Max level shown:
1 [ Nov 12 20:45:23 Enabled. ]
2 [ Nov 12 20:45:23 Rereading configuration. ]
3 [ Nov 12 20:45:24 Rereading configuration. ]
4 [ Nov 12 20:45:31 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Nov 12 20:45:31 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72025-11-12T20:45:32.335ZDEBGnexus: registered DTrace probes
82025-11-12T20:45:32.338ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:80
92025-11-12T20:45:32.350ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102025-11-12T20:45:32.350ZINFOnexus (ServerContext): registering Oso class class = AuditLog file = nexus/auth/src/authz/oso_generic.rs:68
112025-11-12T20:45:32.350ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
122025-11-12T20:45:32.350ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
132025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
142025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
152025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
162025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
172025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
182025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
192025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = VpcList file = nexus/auth/src/authz/oso_generic.rs:68
202025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
212025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
222025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = QuiesceState file = nexus/auth/src/authz/oso_generic.rs:68
232025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
242025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = SiloGroupList file = nexus/auth/src/authz/oso_generic.rs:68
252025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
262025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
272025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = SiloUserSessionList file = nexus/auth/src/authz/oso_generic.rs:68
282025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = SiloUserTokenList file = nexus/auth/src/authz/oso_generic.rs:68
292025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = UpdateTrustRootList file = nexus/auth/src/authz/oso_generic.rs:68
302025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = TargetReleaseConfig file = nexus/auth/src/authz/oso_generic.rs:68
312025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = AlertClassList file = nexus/auth/src/authz/oso_generic.rs:68
322025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = ScimClientBearerTokenList file = nexus/auth/src/authz/oso_generic.rs:68
332025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
342025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
352025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
362025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
372025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = AffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
382025-11-12T20:45:32.351ZINFOnexus (ServerContext): registering Oso class class = AntiAffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
392025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
402025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
412025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
422025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
432025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
442025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = InternetGateway file = nexus/auth/src/authz/oso_generic.rs:68
452025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpPool file = nexus/auth/src/authz/oso_generic.rs:68
462025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpAddress file = nexus/auth/src/authz/oso_generic.rs:68
472025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
482025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
492025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
502025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
512025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
522025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
532025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
542025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
552025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
562025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
572025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
582025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
592025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
602025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
612025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
622025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
632025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
642025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
652025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = SupportBundle file = nexus/auth/src/authz/oso_generic.rs:68
662025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
672025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
682025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
692025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
702025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
712025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = TufTrustRoot file = nexus/auth/src/authz/oso_generic.rs:68
722025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Alert file = nexus/auth/src/authz/oso_generic.rs:68
732025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = AlertReceiver file = nexus/auth/src/authz/oso_generic.rs:68
742025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = WebhookSecret file = nexus/auth/src/authz/oso_generic.rs:68
752025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
762025-11-12T20:45:32.352ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
772025-11-12T20:45:32.353ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
782025-11-12T20:45:32.353ZINFOnexus (ServerContext): registering Oso class class = ScimClientBearerToken file = nexus/auth/src/authz/oso_generic.rs:68
792025-11-12T20:45:32.353ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations; grants project.admin on all projects)\n# - silo.limited-collaborator (grants project.limited-collaborator on all projects)\n# - silo.viewer (can read most resources within the Silo; grants project.viewer)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project, including networking)\n# - project.limited-collaborator (can manage compute resources, but not networking resources)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\n# Note that certain Actors may not ever have any roles assigned to them, like\n# SCIM Actors.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "limited-collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "limited-collaborator";\n\t"limited-collaborator" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif actor.is_user and silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "limited-collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t# Role hierarchy: admin > collaborator > limited-collaborator > viewer\n\t#\n\t# The "limited-collaborator" role can create/modify non-networking\n\t# resources (instances, disks, etc.) but cannot create/modify networking\n\t# infrastructure (VPCs, subnets, routers, internet gateways).\n\t# See nexus/authz-macros for InProjectLimited vs InProjectFull.\n\t"viewer" if "limited-collaborator";\n\t"limited-collaborator" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "limited-collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"limited-collaborator" if "limited-collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if actor.is_user and silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n}\n# We want to allow the user to modify the ssh key but disallow a SCIM IdP token\n# from doing the same.\nhas_permission(actor: AuthenticatedActor, "modify", ssh_key: SshKey)\n\tif actor.is_user and has_permission(actor, "modify", ssh_key.silo_user);\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource ScimClientBearerToken {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n # necessary to authenticate SCIM actors\n\t"read" if "external-authenticator" on "parent_fleet";\n\n\t# Silo-level roles grant privileges for SCIM client tokens.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges for SCIM client tokens.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", scim_client_bearer_token: ScimClientBearerToken)\n\tif scim_client_bearer_token.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: ScimClientBearerToken)\n\tif collection.silo.fleet = fleet;\n\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the quiesce state of a particular Nexus instance.\n#\n# These authz checks must not require the database. We grant this directly to\n# callers of the internal API.\nresource QuiesceState {\n\tpermissions = [ "read", "modify" ];\n}\nhas_permission(USER_INTERNAL_API: AuthenticatedActor, "read", _q: QuiesceState);\nhas_permission(\n USER_INTERNAL_API: AuthenticatedActor,\n "modify",\n _q: QuiesceState\n);\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/update/trust-roots" in the API\nresource UpdateTrustRootList {\n\tpermissions = [ "list_children", "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"list_children" if "viewer" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: UpdateTrustRootList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif actor.is_user and silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for reading and writing the audit log \nresource AuditLog {\n\tpermissions = [\n\t "list_children", # retrieve audit log\n\t "create_child", # create audit log entry\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\n\t# Fleet viewers can read the audit log\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\n# Any actor should be able to write to the audit log because we need to be able\n# to write to the log from any request, authenticated or not. Audit log writes\n# are always a byproduct of other operations: there are no endpoints that allow\n# the user to write to the log deliberately. Note we use AuthenticatedActor\n# because we don't really mean unauthenticated -- in the case of login\n# operations, we use the external authenticator actor that creates the session\n# to authorize the audit log write.\nhas_permission(_actor: AuthenticatedActor, "create_child", _audit_log: AuditLog);\n\nhas_relation(fleet: Fleet, "parent_fleet", audit_log: AuditLog)\n\tif audit_log.fleet = fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Allow silo admins to delete and list user sessions\nresource SiloUserSessionList {\n permissions = [ "modify", "list_children" ];\n relations = { parent_silo: Silo };\n\n # A silo admin can modify (e.g., delete) a user's sessions.\n "modify" if "admin" on "parent_silo";\n\n # A silo admin can list a user's sessions.\n "list_children" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", authn_list: SiloUserSessionList)\n if authn_list.silo_user.silo = silo;\n\n# give users 'modify' and 'list_children' on their own sessions\nhas_permission(actor: AuthenticatedActor, "modify", authn_list: SiloUserSessionList)\n if actor.equals_silo_user(authn_list.silo_user);\nhas_permission(actor: AuthenticatedActor, "list_children", authn_list: SiloUserSessionList)\n if actor.equals_silo_user(authn_list.silo_user);\n\n# Allow silo admins to delete and list user access tokens\nresource SiloUserTokenList {\n permissions = [ "modify", "list_children" ];\n relations = { parent_silo: Silo };\n\n # A silo admin can modify (e.g., delete) a user's tokens.\n "modify" if "admin" on "parent_silo";\n\n # A silo admin can list a user's tokens.\n "list_children" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", authn_list: SiloUserTokenList)\n if authn_list.silo_user.silo = silo;\n\n# give users 'modify' and 'list_children' on their own tokens\nhas_permission(actor: AuthenticatedActor, "modify", authn_list: SiloUserTokenList)\n if actor.equals_silo_user(authn_list.silo_user);\nhas_permission(actor: AuthenticatedActor, "list_children", authn_list: SiloUserTokenList)\n if actor.equals_silo_user(authn_list.silo_user);\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# Grant SCIM IdP actors the permissions they need on users.\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if actor.is_scim_idp and silo_user.silo in actor.silo;\nhas_permission(actor: AuthenticatedActor, "create_child", silo_user_list: SiloUserList)\n\tif actor.is_scim_idp and silo_user_list.silo in actor.silo;\nhas_permission(actor: AuthenticatedActor, "modify", silo_user: SiloUser)\n\tif actor.is_scim_idp and silo_user.silo in actor.silo;\nhas_permission(actor: AuthenticatedActor, "list_children", silo_user_list: SiloUserList)\n if actor.is_scim_idp and silo_user_list.silo in actor.silo;\n\n# Describes the policy for creating and managing Silo groups (mostly intended\n# for API-managed groups)\nresource SiloGroupList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the groups in the\n\t# Silo) can see the groups in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's groups. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloGroupList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloGroupList)\n\tif collection.silo.fleet = fleet;\n\n# Grant SCIM IdP actors the permissions they need on groups.\nhas_permission(actor: AuthenticatedActor, "read", silo_group: SiloGroup)\n if actor.is_scim_idp and silo_group.silo in actor.silo;\nhas_permission(actor: AuthenticatedActor, "create_child", silo_group_list: SiloGroupList)\n\tif actor.is_scim_idp and silo_group_list.silo in actor.silo;\nhas_permission(actor: AuthenticatedActor, "modify", silo_group: SiloGroup)\n\tif actor.is_scim_idp and silo_group.silo in actor.silo;\nhas_permission(actor: AuthenticatedActor, "list_children", silo_group_list: SiloGroupList)\n if actor.is_scim_idp and silo_group_list.silo in actor.silo;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest)\n if actor.is_user;\nhas_permission(actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest)\n if actor.is_user;\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_alert_receiver: AlertReceiver };\n\n\t"read" if "read" on "parent_alert_receiver";\n\t"modify" if "modify" on "parent_alert_receiver";\n}\n\nhas_relation(rx: AlertReceiver, "parent_alert_receiver", secret: WebhookSecret)\n\tif secret.alert_receiver = rx;\n\nresource AlertClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: AlertClassList)\n\tif collection.fleet = fleet;\n\nresource ScimClientBearerTokenList {\n\tpermissions = [ "create_child", "list_children" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges for SCIM client tokens.\n # These are all admin because being able to create these tokens would allow\n # a user to grant themselves admin by modifying group membership through SCIM calls\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_silo";\n\n # Fleet-level roles also grant privileges for SCIM client tokens, for\n # configuration before silo admins are present.\n\t"create_child" if "admin" on "parent_fleet";\n\t"list_children" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", scim_client_bearer_token_list: ScimClientBearerTokenList)\n\tif scim_client_bearer_token_list.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: ScimClientBearerTokenList)\n\tif collection.silo.fleet = fleet;\n\n# VpcList is a synthetic resource for controlling VPC creation.\n# Unlike other project resources, VPC creation requires the full "collaborator"\n# role rather than "limited-collaborator", enforcing the networking restriction.\n# This allows organizations to restrict who can reconfigure the network topology\n# while still allowing users with limited-collaborator to work with compute\n# resources (instances, disks, etc.) within the existing network.\nresource VpcList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { containing_project: Project };\n\n\t"list_children" if "read" on "containing_project";\n\t"create_child" if "collaborator" on "containing_project";\n}\nhas_relation(project: Project, "containing_project", collection: VpcList)\n\tif collection.project = project;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "limited-collaborator" on "containing_project";\n "create_child" if "limited-collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource TufTrustRoot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufTrustRoot)\n if child.fleet = fleet;\n \n\n resource Alert {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Alert)\n if child.fleet = fleet;\n \n\n resource AlertReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AlertReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n \n file = nexus/auth/src/authz/oso_generic.rs:89
802025-11-12T20:45:32.374ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:229
812025-11-12T20:45:32.379ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:144
822025-11-12T20:45:32.381ZINFOnexus (ServerContext): Setting up qorb database pool from DNS dns_addrs = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = nexus/src/context.rs:285
832025-11-12T20:45:32.397ZDEBGnexus (ServerContext): registered USDT probes
842025-11-12T20:45:32.774ZINFOnexus (datastore): Database schema version is up to date desired_version = 207.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:409 found_version = 207.0.0
852025-11-12T20:45:32.818ZWARNnexus (datastore): No db_metadata_nexus records exist - skipping access check explanation = This is expected during initial deployment or before migration file = nexus/db-queries/src/db/datastore/db_metadata.rs:337 nexus_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 (omicron_zone)
862025-11-12T20:45:32.818ZINFOnexus: Datastore is ready for usage file = nexus/db-queries/src/db/datastore/mod.rs:316
872025-11-12T20:45:32.821ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:813 sec_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
882025-11-12T20:45:32.902ZDEBGnexus (clickhouse-client): registered USDT probes id = 66a256d7-6387-47c6-9acf-2d537941b25f
892025-11-12T20:45:32.947ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
902025-11-12T20:45:32.975ZINFOnexus (MgsUpdateDriver): starting MgsUpdateDriver file = nexus/mgs-updates/src/driver.rs:146
912025-11-12T20:45:32.981ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
922025-11-12T20:45:32.985ZDEBGnexus (DataLoader): attempting to create built-in users
932025-11-12T20:45:32.990ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
942025-11-12T20:45:32.990ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
952025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
962025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
972025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
982025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
992025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1002025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1012025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /disks/{disk_id} versions = all
1022025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/collectors versions = all
1032025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1042025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /metrics/producers versions = all
1052025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1062025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /probes/{sled} versions = all
1072025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /refresh-vpc-routes versions = all
1082025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1092025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1102025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /switch/{switch_id} versions = all
1112025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = GET path = /v1/ping versions = all
1122025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1132025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::c]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1142025-11-12T20:45:33.041ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12221
1152025-11-12T20:45:33.041ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12221
1162025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /bgtasks versions = all
1172025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /bgtasks/activate versions = all
1182025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1192025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /clickhouse/policy versions = all
1202025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /clickhouse/policy versions = all
1212025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /debug/fetch-omdb-binary versions = all
1222025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /demo-saga versions = all
1232025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1242025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /deployment/blueprints/all versions = all
1252025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1262025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1272025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /deployment/blueprints/import versions = all
1282025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /deployment/blueprints/regenerate versions = all
1292025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /deployment/blueprints/target versions = all
1302025-11-12T20:45:33.055ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /deployment/blueprints/target versions = all
1312025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = PUT path = /deployment/blueprints/target/enabled versions = all
1322025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /deployment/reconfigurator-config versions = all
1332025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /deployment/reconfigurator-config versions = all
1342025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /deployment/reconfigurator-config/{version} versions = all
1352025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /deployment/update-status versions = all
1362025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /experimental/v1/system/support-bundles versions = all
1372025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /experimental/v1/system/support-bundles versions = all
1382025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1392025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1402025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1412025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1422025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1432025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1442025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1452025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
1462025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /instances/{instance_id}/migrate versions = all
1472025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /mgs-updates versions = all
1482025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /oximeter/read-policy versions = all
1492025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /oximeter/read-policy versions = all
1502025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /physical-disk/expunge versions = all
1512025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /quiesce versions = all
1522025-11-12T20:45:33.056ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /quiesce versions = all
1532025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1542025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /sagas versions = all
1552025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /sagas/{saga_id} versions = all
1562025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /sleds/add versions = all
1572025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = POST path = /sleds/expunge versions = all
1582025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /sleds/uninitialized versions = all
1592025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): registered endpoint local_addr = [fd00:1122:3344:101::c]:12232 method = GET path = /v1/ping versions = all
1602025-11-12T20:45:33.057ZINFOnexus (dropshot_lockstep): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12232
1612025-11-12T20:45:33.057ZDEBGnexus (dropshot_lockstep): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12232
1622025-11-12T20:45:33.057ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1632025-11-12T20:45:33.057ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
1642025-11-12T20:45:33.097ZWARNnexus: Cannot look up rack: Object (of type ById(21dec677-5bdd-45c9-88cb-9a589af920a7)) not found: rack file = nexus/src/app/rack.rs:785
1652025-11-12T20:45:33.224ZINFOnexus (DataLoader): created 0 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:907
1662025-11-12T20:45:33.224ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1672025-11-12T20:45:33.225ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1682025-11-12T20:45:33.225ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1692025-11-12T20:45:33.225ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1702025-11-12T20:45:33.225ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1712025-11-12T20:45:33.917ZINFOnexus (DataLoader): created 0 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:55
1722025-11-12T20:45:33.917ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1732025-11-12T20:45:33.919ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1742025-11-12T20:45:33.919ZDEBGnexus (DataLoader): attempting to create built-in silos
1752025-11-12T20:45:33.919ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1762025-11-12T20:45:33.919ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1772025-11-12T20:45:34.016ZINFOnexus (DataLoader): created 0 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:102
1782025-11-12T20:45:34.019ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1792025-11-12T20:45:34.019ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1802025-11-12T20:45:34.031ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1812025-11-12T20:45:34.032ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1822025-11-12T20:45:34.032ZDEBGnexus (DataLoader): attempting to create built-in projects
1832025-11-12T20:45:34.034ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1842025-11-12T20:45:34.035ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1852025-11-12T20:45:34.046ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1862025-11-12T20:45:34.046ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1872025-11-12T20:45:34.109ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1882025-11-12T20:45:34.109ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1892025-11-12T20:45:34.153ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1902025-11-12T20:45:34.157ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1912025-11-12T20:45:34.157ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1922025-11-12T20:45:34.157ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1932025-11-12T20:45:34.206ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1942025-11-12T20:45:34.207ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1952025-11-12T20:45:34.209ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1962025-11-12T20:45:34.211ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1972025-11-12T20:45:34.212ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1982025-11-12T20:45:34.212ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
1992025-11-12T20:45:34.265ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
2002025-11-12T20:45:34.265ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2012025-11-12T20:45:34.266ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2022025-11-12T20:45:34.266ZDEBGnexus (DataLoader): attempting to create built-in VPCs
2032025-11-12T20:45:34.268ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2042025-11-12T20:45:34.268ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2052025-11-12T20:45:34.281ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2062025-11-12T20:45:34.282ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2072025-11-12T20:45:34.291ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2082025-11-12T20:45:34.291ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2092025-11-12T20:45:34.293ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2102025-11-12T20:45:34.293ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2112025-11-12T20:45:34.294ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2122025-11-12T20:45:34.295ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2132025-11-12T20:45:34.334ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2142025-11-12T20:45:34.337ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2152025-11-12T20:45:34.337ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2162025-11-12T20:45:34.337ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2172025-11-12T20:45:34.339ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2182025-11-12T20:45:34.339ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2192025-11-12T20:45:34.340ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2202025-11-12T20:45:34.340ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2212025-11-12T20:45:34.342ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2222025-11-12T20:45:34.346ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2232025-11-12T20:45:34.346ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2242025-11-12T20:45:34.348ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2252025-11-12T20:45:34.389ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2262025-11-12T20:45:34.389ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2272025-11-12T20:45:34.391ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2282025-11-12T20:45:34.391ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2292025-11-12T20:45:34.393ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2302025-11-12T20:45:34.393ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2312025-11-12T20:45:34.395ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2322025-11-12T20:45:34.398ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2332025-11-12T20:45:34.398ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2342025-11-12T20:45:34.398ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2352025-11-12T20:45:34.407ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2362025-11-12T20:45:34.407ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2372025-11-12T20:45:34.410ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2382025-11-12T20:45:34.410ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2392025-11-12T20:45:34.424ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2402025-11-12T20:45:34.425ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2412025-11-12T20:45:34.426ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2422025-11-12T20:45:34.426ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2432025-11-12T20:45:34.427ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2442025-11-12T20:45:34.428ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2452025-11-12T20:45:34.430ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2462025-11-12T20:45:34.430ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2472025-11-12T20:45:34.431ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2482025-11-12T20:45:34.431ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2492025-11-12T20:45:34.432ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2502025-11-12T20:45:34.436ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2512025-11-12T20:45:34.436ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2522025-11-12T20:45:34.436ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2532025-11-12T20:45:34.438ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2542025-11-12T20:45:34.438ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2552025-11-12T20:45:34.439ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2562025-11-12T20:45:34.439ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2572025-11-12T20:45:34.440ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2582025-11-12T20:45:34.444ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2592025-11-12T20:45:34.444ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2602025-11-12T20:45:34.445ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2612025-11-12T20:45:34.535ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2622025-11-12T20:45:34.535ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2632025-11-12T20:45:34.536ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2642025-11-12T20:45:34.536ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2652025-11-12T20:45:34.538ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2662025-11-12T20:45:34.538ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2672025-11-12T20:45:34.540ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2682025-11-12T20:45:34.544ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
2692025-11-12T20:45:34.544ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2702025-11-12T20:45:34.544ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2712025-11-12T20:45:34.589ZDEBGnexus (DataLoader): attempting to create built-in VPC firewall rules
2722025-11-12T20:45:34.589ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2732025-11-12T20:45:34.589ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2742025-11-12T20:45:34.591ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2752025-11-12T20:45:34.591ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2762025-11-12T20:45:34.593ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2772025-11-12T20:45:34.593ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2782025-11-12T20:45:34.595ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2792025-11-12T20:45:34.595ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2802025-11-12T20:45:34.597ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2812025-11-12T20:45:34.597ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2822025-11-12T20:45:34.599ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2832025-11-12T20:45:34.599ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2842025-11-12T20:45:34.601ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2852025-11-12T20:45:34.604ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2862025-11-12T20:45:34.604ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2872025-11-12T20:45:34.604ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2882025-11-12T20:45:34.605ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2892025-11-12T20:45:34.605ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2902025-11-12T20:45:34.607ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2912025-11-12T20:45:34.607ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2922025-11-12T20:45:34.609ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2932025-11-12T20:45:34.614ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2942025-11-12T20:45:34.614ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2952025-11-12T20:45:34.615ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2962025-11-12T20:45:34.647ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2972025-11-12T20:45:34.647ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
2982025-11-12T20:45:34.648ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2992025-11-12T20:45:34.649ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3002025-11-12T20:45:34.650ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3012025-11-12T20:45:34.650ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3022025-11-12T20:45:34.651ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3032025-11-12T20:45:34.655ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3042025-11-12T20:45:34.655ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3052025-11-12T20:45:34.655ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3062025-11-12T20:45:34.727ZINFOnexus (DataLoader): created built-in VPC firewall rules file = nexus/db-queries/src/db/datastore/vpc.rs:301
3072025-11-12T20:45:34.727ZDEBGnexus (DataLoader): attempting to create built-in VPC Subnets
3082025-11-12T20:45:34.727ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3092025-11-12T20:45:34.728ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3102025-11-12T20:45:34.740ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3112025-11-12T20:45:34.741ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3122025-11-12T20:45:34.743ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3132025-11-12T20:45:34.743ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3142025-11-12T20:45:34.744ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3152025-11-12T20:45:34.745ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3162025-11-12T20:45:34.746ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3172025-11-12T20:45:34.746ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3182025-11-12T20:45:34.748ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3192025-11-12T20:45:34.748ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3202025-11-12T20:45:34.749ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3212025-11-12T20:45:34.752ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3222025-11-12T20:45:34.752ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3232025-11-12T20:45:34.752ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3242025-11-12T20:45:34.755ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3252025-11-12T20:45:34.755ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3262025-11-12T20:45:34.756ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3272025-11-12T20:45:34.756ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3282025-11-12T20:45:34.758ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3292025-11-12T20:45:34.758ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3302025-11-12T20:45:34.759ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3312025-11-12T20:45:34.759ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3322025-11-12T20:45:34.761ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3332025-11-12T20:45:34.761ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3342025-11-12T20:45:34.762ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3352025-11-12T20:45:34.762ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3362025-11-12T20:45:34.763ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3372025-11-12T20:45:34.768ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ById(001de000-c470-4000-8000-000000000001) } result = Ok(())
3382025-11-12T20:45:34.768ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3392025-11-12T20:45:34.768ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3402025-11-12T20:45:34.770ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3412025-11-12T20:45:34.770ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3422025-11-12T20:45:34.772ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3432025-11-12T20:45:34.772ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3442025-11-12T20:45:34.773ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3452025-11-12T20:45:34.774ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3462025-11-12T20:45:34.775ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3472025-11-12T20:45:34.775ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3482025-11-12T20:45:34.783ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3492025-11-12T20:45:34.783ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3502025-11-12T20:45:34.785ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3512025-11-12T20:45:34.785ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3522025-11-12T20:45:34.787ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3532025-11-12T20:45:34.792ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
3542025-11-12T20:45:34.792ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3552025-11-12T20:45:34.792ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3562025-11-12T20:45:34.794ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3572025-11-12T20:45:34.795ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3582025-11-12T20:45:34.799ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3592025-11-12T20:45:34.799ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3602025-11-12T20:45:34.800ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3612025-11-12T20:45:34.800ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3622025-11-12T20:45:34.802ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3632025-11-12T20:45:34.802ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3642025-11-12T20:45:34.803ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3652025-11-12T20:45:34.803ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3662025-11-12T20:45:34.804ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3672025-11-12T20:45:34.805ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3682025-11-12T20:45:34.806ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3692025-11-12T20:45:34.811ZDEBGnexus (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ById(001de000-c470-4000-8000-000000000003) } result = Ok(())
3702025-11-12T20:45:34.811ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3712025-11-12T20:45:34.811ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3722025-11-12T20:45:34.815ZINFOnexus (DataLoader): created built-in services vpc subnets file = nexus/db-queries/src/db/datastore/vpc.rs:367
3732025-11-12T20:45:34.815ZINFOnexus (DataLoader): created built-in services vpc file = nexus/db-queries/src/db/datastore/vpc.rs:233
3742025-11-12T20:45:34.815ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3752025-11-12T20:45:34.816ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3762025-11-12T20:45:34.816ZDEBGnexus (DataLoader): attempting to create silo users
3772025-11-12T20:45:34.816ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3782025-11-12T20:45:34.816ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3792025-11-12T20:45:34.822ZINFOnexus (DataLoader): created 0 silo users file = nexus/db-queries/src/db/datastore/silo_user.rs:932
3802025-11-12T20:45:34.822ZDEBGnexus (DataLoader): attempting to create silo user role assignments
3812025-11-12T20:45:34.822ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3822025-11-12T20:45:34.822ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3832025-11-12T20:45:34.824ZINFOnexus (DataLoader): created 0 silo user role assignments file = nexus/db-queries/src/db/datastore/silo_user.rs:959
3842025-11-12T20:45:34.824ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3852025-11-12T20:45:34.824ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3862025-11-12T20:45:34.833ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3872025-11-12T20:45:34.833ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3882025-11-12T20:45:34.847ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3892025-11-12T20:45:34.847ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3902025-11-12T20:45:34.854ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3912025-11-12T20:45:34.856ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = IpPoolList result = Ok(())
3922025-11-12T20:45:34.856ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3932025-11-12T20:45:34.856ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3942025-11-12T20:45:35.017ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3952025-11-12T20:45:35.018ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
3962025-11-12T20:45:35.019ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3972025-11-12T20:45:35.021ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = IpPoolList result = Ok(())
3982025-11-12T20:45:35.021ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
3992025-11-12T20:45:35.021ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001 (built_in_user), .. }) resource = Database result = Ok(())
4002025-11-12T20:45:35.098ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
4012025-11-12T20:45:35.098ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
4022025-11-12T20:45:35.100ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
4032025-11-12T20:45:35.100ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
4042025-11-12T20:45:35.104ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4052025-11-12T20:45:35.107ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
4062025-11-12T20:45:35.107ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:779
4072025-11-12T20:45:35.268ZINFOnexus: populate complete file = nexus/src/app/mod.rs:576
4082025-11-12T20:45:35.268ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:607
4092025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
4102025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
4112025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
4122025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
4132025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
4142025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
4152025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
4162025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
4172025-11-12T20:45:35.276ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 1 reason = Timeout
4182025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4192025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4202025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4212025-11-12T20:45:35.277ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:35:35.277163031 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4222025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
4232025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
4242025-11-12T20:45:35.277ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = no blueprint
4252025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
4262025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 1 reason = Timeout
4272025-11-12T20:45:35.277ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4282025-11-12T20:45:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
4292025-11-12T20:45:35.277ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
4302025-11-12T20:45:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4312025-11-12T20:45:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4322025-11-12T20:45:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 2.376522ms iteration = 1 status = {"error":"no blueprint"}
4332025-11-12T20:45:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 2.30207ms iteration = 1 status = {"error":"no config nor servers"}
4342025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
4352025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 1 reason = Timeout
4362025-11-12T20:45:35.290ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
4372025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4382025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 1 reason = Timeout
4392025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
4402025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
4412025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4422025-11-12T20:45:35.290ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
4432025-11-12T20:45:35.290ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
4442025-11-12T20:45:35.290ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
4452025-11-12T20:45:35.290ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 1 reason = Timeout
4462025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 2.376012ms iteration = 1 status = {"error":"no config nor servers"}
4472025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 1 reason = Timeout
4482025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
4492025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
4502025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4512025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
4522025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
4532025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
4542025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 1 reason = Timeout
4552025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
4562025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
4572025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
4582025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4592025-11-12T20:45:35.291ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:54 reason = no blueprint
4602025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 12.374561ms iteration = 1 status = {"error":"no blueprint"}
4612025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4622025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): reconfigurator config not yet loaded; doing nothing background_task = blueprint_planner
4632025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 12.631827ms iteration = 1 status = "Disabled"
4642025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
4652025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
4662025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4672025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
4682025-11-12T20:45:35.291ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
4692025-11-12T20:45:35.291ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4702025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
4712025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
4722025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4732025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4742025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
4752025-11-12T20:45:35.293ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
4762025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
4772025-11-12T20:45:35.293ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:72
4782025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
4792025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4802025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 1 reason = Timeout
4812025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4822025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4832025-11-12T20:45:35.293ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 15.128081ms iteration = 1 status = {"error":"inventory collection is None"}
4842025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
4852025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
4862025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
4872025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
4882025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
4892025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4902025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
4912025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4922025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4932025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
4942025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4952025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
4962025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
4972025-11-12T20:45:35.294ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4982025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
4992025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5002025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5012025-11-12T20:45:35.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5022025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5032025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5042025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
5052025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 1 reason = Timeout
5062025-11-12T20:45:35.297ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
5072025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 19.040313ms iteration = 1 status = {"error":"no blueprint"}
5082025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 1 reason = Timeout
5092025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
5102025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
5112025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
5122025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
5132025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
5142025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
5152025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5162025-11-12T20:45:35.297ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
5172025-11-12T20:45:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
5182025-11-12T20:45:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
5192025-11-12T20:45:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
5202025-11-12T20:45:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5212025-11-12T20:45:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
5222025-11-12T20:45:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
5232025-11-12T20:45:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
5242025-11-12T20:45:35.300ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
5252025-11-12T20:45:35.300ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
5262025-11-12T20:45:35.300ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5272025-11-12T20:45:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5282025-11-12T20:45:35.301ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 930, tv_nsec: 796380131 } })
5292025-11-12T20:45:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5302025-11-12T20:45:35.301ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 930, tv_nsec: 797572812 } })
5312025-11-12T20:45:35.302ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 930, tv_nsec: 797903401 } })
5322025-11-12T20:45:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5332025-11-12T20:45:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
5342025-11-12T20:45:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5352025-11-12T20:45:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5362025-11-12T20:45:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
5372025-11-12T20:45:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5382025-11-12T20:45:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
5392025-11-12T20:45:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
5402025-11-12T20:45:35.318ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
5412025-11-12T20:45:35.318ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 42.617313ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
5422025-11-12T20:45:35.318ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
5432025-11-12T20:45:35.318ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
5442025-11-12T20:45:35.318ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 29.241\u{b5}s iteration = 2 status = {"error":"no config"}
5452025-11-12T20:45:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5462025-11-12T20:45:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5472025-11-12T20:45:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5482025-11-12T20:45:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
5492025-11-12T20:45:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5502025-11-12T20:45:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
5512025-11-12T20:45:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
5522025-11-12T20:45:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
5532025-11-12T20:45:35.320ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 930, tv_nsec: 816275836 } })
5542025-11-12T20:45:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5552025-11-12T20:45:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5562025-11-12T20:45:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
5572025-11-12T20:45:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5582025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
5592025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5602025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
5612025-11-12T20:45:35.321ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
5622025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5632025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5642025-11-12T20:45:35.321ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
5652025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 45.574419ms iteration = 1 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
5662025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
5672025-11-12T20:45:35.321ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
5682025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.6\u{b5}s iteration = 2 status = {"error":"no config"}
5692025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5702025-11-12T20:45:35.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
5712025-11-12T20:45:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
5722025-11-12T20:45:35.326ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:35:35.277163031 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5732025-11-12T20:45:35.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5742025-11-12T20:45:35.329ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 53.383372ms iteration = 1 status = {"expiration":"2025-11-12T20:35:35.277163031Z","pruned":[]}
5752025-11-12T20:45:35.329ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 50.735673ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
5762025-11-12T20:45:35.329ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5772025-11-12T20:45:35.330ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782025-11-12T20:45:35.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5792025-11-12T20:45:35.333ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5802025-11-12T20:45:35.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5812025-11-12T20:45:35.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5822025-11-12T20:45:35.334ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5832025-11-12T20:45:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
5842025-11-12T20:45:35.335ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
5852025-11-12T20:45:35.335ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
5862025-11-12T20:45:35.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
5872025-11-12T20:45:35.358ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 930, tv_nsec: 853859689 } })
5882025-11-12T20:45:35.359ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5892025-11-12T20:45:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5902025-11-12T20:45:35.378ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 102.235316ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5912025-11-12T20:45:35.397ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
5922025-11-12T20:45:35.400ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 121.608987ms iteration = 1 status = {"by_rx":{},"error":null}
5932025-11-12T20:45:35.400ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5942025-11-12T20:45:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5952025-11-12T20:45:35.404ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5962025-11-12T20:45:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5972025-11-12T20:45:35.425ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
5982025-11-12T20:45:35.426ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5992025-11-12T20:45:35.428ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = None file = nexus/types/src/quiesce.rs:525 generation = 1
6002025-11-12T20:45:35.428ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 150.34913ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
6012025-11-12T20:45:35.429ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6022025-11-12T20:45:35.430ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6032025-11-12T20:45:35.430ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6042025-11-12T20:45:35.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6052025-11-12T20:45:35.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6062025-11-12T20:45:35.431ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6072025-11-12T20:45:35.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
6082025-11-12T20:45:35.434ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
6092025-11-12T20:45:35.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
6102025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
6112025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
6122025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6132025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6142025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
6152025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
6162025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
6172025-11-12T20:45:35.436ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
6182025-11-12T20:45:35.437ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
6192025-11-12T20:45:35.437ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
6202025-11-12T20:45:35.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
6212025-11-12T20:45:35.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
6222025-11-12T20:45:35.441ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6232025-11-12T20:45:35.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6242025-11-12T20:45:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6252025-11-12T20:45:35.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6262025-11-12T20:45:35.445ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6272025-11-12T20:45:35.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6282025-11-12T20:45:35.457ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6292025-11-12T20:45:35.460ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6302025-11-12T20:45:35.460ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6312025-11-12T20:45:35.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6322025-11-12T20:45:35.461ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6332025-11-12T20:45:35.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6342025-11-12T20:45:35.463ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6352025-11-12T20:45:35.463ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
6362025-11-12T20:45:35.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6372025-11-12T20:45:35.465ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6382025-11-12T20:45:35.465ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
6392025-11-12T20:45:35.465ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6402025-11-12T20:45:35.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6412025-11-12T20:45:35.466ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4015eb4-0e25-4131-ad4a-214d1d54af09", "content-length": "78", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
6422025-11-12T20:45:35.467ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6432025-11-12T20:45:35.469ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
6442025-11-12T20:45:35.469ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6452025-11-12T20:45:35.469ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6462025-11-12T20:45:35.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6472025-11-12T20:45:35.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6482025-11-12T20:45:35.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6492025-11-12T20:45:35.470ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6502025-11-12T20:45:35.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6512025-11-12T20:45:35.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
6522025-11-12T20:45:35.471ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
6532025-11-12T20:45:35.471ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
6542025-11-12T20:45:35.471ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
6552025-11-12T20:45:35.472ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 930, tv_nsec: 968123006 } })
6562025-11-12T20:45:35.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
6572025-11-12T20:45:35.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
6582025-11-12T20:45:35.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
6592025-11-12T20:45:35.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
6602025-11-12T20:45:35.472ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
6612025-11-12T20:45:35.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
6622025-11-12T20:45:35.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6632025-11-12T20:45:35.475ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6642025-11-12T20:45:35.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6652025-11-12T20:45:35.476ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6662025-11-12T20:45:35.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6672025-11-12T20:45:35.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6682025-11-12T20:45:35.480ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692025-11-12T20:45:35.481ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
6702025-11-12T20:45:35.481ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6712025-11-12T20:45:35.481ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
6722025-11-12T20:45:35.481ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "63607f51-adf4-45e1-8e23-79815c45ab61", "content-length": "26", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
6732025-11-12T20:45:35.481ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6742025-11-12T20:45:35.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
6752025-11-12T20:45:35.483ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6762025-11-12T20:45:35.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
6772025-11-12T20:45:35.484ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
6782025-11-12T20:45:35.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
6792025-11-12T20:45:35.491ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6802025-11-12T20:45:35.491ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6812025-11-12T20:45:35.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6822025-11-12T20:45:35.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
6832025-11-12T20:45:35.501ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6842025-11-12T20:45:35.501ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6852025-11-12T20:45:35.501ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 222.99382ms iteration = 1 status = {}
6862025-11-12T20:45:35.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6872025-11-12T20:45:35.507ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6882025-11-12T20:45:35.507ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:82 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
6892025-11-12T20:45:35.507ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 228.638087ms iteration = 1 status = {"error":"rack not yet initialized: 21dec677-5bdd-45c9-88cb-9a589af920a7"}
6902025-11-12T20:45:35.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6912025-11-12T20:45:35.511ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6922025-11-12T20:45:35.511ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6932025-11-12T20:45:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6942025-11-12T20:45:35.512ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
6952025-11-12T20:45:35.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
6962025-11-12T20:45:35.513ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
6972025-11-12T20:45:35.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
6982025-11-12T20:45:35.515ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
6992025-11-12T20:45:35.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
7002025-11-12T20:45:35.516ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7012025-11-12T20:45:35.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
7022025-11-12T20:45:35.517ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
7032025-11-12T20:45:35.517ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
7042025-11-12T20:45:35.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
7052025-11-12T20:45:35.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
7062025-11-12T20:45:35.519ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
7072025-11-12T20:45:35.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
7082025-11-12T20:45:35.519ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
7092025-11-12T20:45:35.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 242.012783ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
7102025-11-12T20:45:35.522ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 244.401036ms iteration = 1 status = "NoCollections"
7112025-11-12T20:45:35.523ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
7122025-11-12T20:45:35.523ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
7132025-11-12T20:45:35.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
7142025-11-12T20:45:35.523ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 239.445346ms iteration = 1 status = "NoSitrep"
7152025-11-12T20:45:35.523ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
7162025-11-12T20:45:35.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
7172025-11-12T20:45:35.525ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7182025-11-12T20:45:35.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7192025-11-12T20:45:35.526ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
7202025-11-12T20:45:35.526ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
7212025-11-12T20:45:35.526ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 247.485134ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
7222025-11-12T20:45:35.527ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
7232025-11-12T20:45:35.528ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7242025-11-12T20:45:35.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
7252025-11-12T20:45:35.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
7262025-11-12T20:45:35.531ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 2 reason = Dependency
7272025-11-12T20:45:35.531ZWARNnexus (BackgroundTasks): blueprint planning skipped background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:105 reason = no target blueprint loaded
7282025-11-12T20:45:35.531ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
7292025-11-12T20:45:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 19.38\u{b5}s iteration = 2 status = {"Error":"no target blueprint to use as parent for planning"}
7302025-11-12T20:45:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 253.144181ms iteration = 1 status = {"config_updated":true}
7312025-11-12T20:45:35.539ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:71
7322025-11-12T20:45:35.539ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 261.007284ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
7332025-11-12T20:45:35.547ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 268.103608ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
7342025-11-12T20:45:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7352025-11-12T20:45:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7362025-11-12T20:45:35.570ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
7372025-11-12T20:45:35.570ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 294.342758ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
7382025-11-12T20:45:35.571ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 292.498389ms iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
7392025-11-12T20:45:35.572ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
7402025-11-12T20:45:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
7412025-11-12T20:45:35.574ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7422025-11-12T20:45:35.574ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7432025-11-12T20:45:35.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7442025-11-12T20:45:35.576ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 297.356585ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
7452025-11-12T20:45:35.577ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462025-11-12T20:45:35.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7472025-11-12T20:45:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7482025-11-12T20:45:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
7492025-11-12T20:45:35.580ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
7502025-11-12T20:45:35.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
7512025-11-12T20:45:35.581ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7522025-11-12T20:45:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7532025-11-12T20:45:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
7542025-11-12T20:45:35.583ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
7552025-11-12T20:45:35.583ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 307.654381ms iteration = 1 status = {}
7562025-11-12T20:45:35.584ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7572025-11-12T20:45:35.584ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 308.507424ms iteration = 1 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
7582025-11-12T20:45:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7592025-11-12T20:45:35.587ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7602025-11-12T20:45:35.588ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
7612025-11-12T20:45:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
7622025-11-12T20:45:35.589ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7632025-11-12T20:45:35.590ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7642025-11-12T20:45:35.591ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
7652025-11-12T20:45:35.591ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 301.204795ms iteration = 1 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
7662025-11-12T20:45:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
7672025-11-12T20:45:35.593ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
7682025-11-12T20:45:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7692025-11-12T20:45:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7702025-11-12T20:45:35.594ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7712025-11-12T20:45:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7722025-11-12T20:45:35.595ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 931, tv_nsec: 90918854 } })
7732025-11-12T20:45:35.596ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7742025-11-12T20:45:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7752025-11-12T20:45:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
7762025-11-12T20:45:35.597ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
7772025-11-12T20:45:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
7782025-11-12T20:45:35.598ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7792025-11-12T20:45:35.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7802025-11-12T20:45:35.603ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7812025-11-12T20:45:35.603ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
7822025-11-12T20:45:35.603ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7832025-11-12T20:45:35.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
7842025-11-12T20:45:35.604ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7852025-11-12T20:45:35.614ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
7862025-11-12T20:45:35.614ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 338.630603ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
7872025-11-12T20:45:35.616ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
7882025-11-12T20:45:35.616ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 337.438322ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
7892025-11-12T20:45:35.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7902025-11-12T20:45:35.617ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7912025-11-12T20:45:35.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7922025-11-12T20:45:35.617ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:114 status = TufRepoPrunerStatus { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
7932025-11-12T20:45:35.617ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 338.726756ms iteration = 1 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"warnings":[]}
7942025-11-12T20:45:35.618ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 339.885986ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7952025-11-12T20:45:35.619ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7962025-11-12T20:45:35.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7972025-11-12T20:45:35.623ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7982025-11-12T20:45:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7992025-11-12T20:45:35.624ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
8002025-11-12T20:45:35.625ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
8012025-11-12T20:45:35.625ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
8022025-11-12T20:45:35.625ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 346.364714ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
8032025-11-12T20:45:35.626ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8042025-11-12T20:45:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8052025-11-12T20:45:35.627ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
8062025-11-12T20:45:35.628ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "be1a80a8-ee1c-4c8c-bda6-b33bb3f04929", "content-length": "81", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
8072025-11-12T20:45:35.628ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
8082025-11-12T20:45:35.628ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
8092025-11-12T20:45:35.629ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6867b429-2774-4b15-88e0-1e5f9e037b12", "content-length": "81", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
8102025-11-12T20:45:35.631ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
8112025-11-12T20:45:35.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
8122025-11-12T20:45:35.633ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8132025-11-12T20:45:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8142025-11-12T20:45:35.636ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 357.886821ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
8152025-11-12T20:45:35.637ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8162025-11-12T20:45:35.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8172025-11-12T20:45:35.639ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8182025-11-12T20:45:35.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8192025-11-12T20:45:35.646ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8202025-11-12T20:45:35.647ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
8212025-11-12T20:45:35.647ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
8222025-11-12T20:45:35.647ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdc99ea6-8a96-4303-9bf5-ffabeef4ab4a", "content-length": "81", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
8232025-11-12T20:45:35.647ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
8242025-11-12T20:45:35.647ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 371.788401ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
8252025-11-12T20:45:35.649ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 370.57661ms iteration = 1 status = {"disabled":false,"errors":[],"sps":[]}
8262025-11-12T20:45:35.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8272025-11-12T20:45:35.653ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8282025-11-12T20:45:35.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8292025-11-12T20:45:35.655ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8302025-11-12T20:45:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8312025-11-12T20:45:35.656ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8322025-11-12T20:45:35.656ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8332025-11-12T20:45:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
8342025-11-12T20:45:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8352025-11-12T20:45:35.658ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8362025-11-12T20:45:35.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8372025-11-12T20:45:35.663ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 384.788098ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
8382025-11-12T20:45:35.669ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
8392025-11-12T20:45:35.669ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 391.366568ms iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
8402025-11-12T20:45:35.673ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8412025-11-12T20:45:35.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8422025-11-12T20:45:35.679ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8432025-11-12T20:45:35.679ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8442025-11-12T20:45:35.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
8452025-11-12T20:45:35.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8462025-11-12T20:45:35.682ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8472025-11-12T20:45:35.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8482025-11-12T20:45:35.682ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8492025-11-12T20:45:35.683ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 404.493508ms iteration = 1 status = {}
8502025-11-12T20:45:35.684ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8512025-11-12T20:45:35.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8522025-11-12T20:45:35.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8532025-11-12T20:45:35.686ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8542025-11-12T20:45:35.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
8552025-11-12T20:45:35.686ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8562025-11-12T20:45:35.688ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8572025-11-12T20:45:35.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8582025-11-12T20:45:35.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8592025-11-12T20:45:35.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
8602025-11-12T20:45:35.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8612025-11-12T20:45:35.693ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8622025-11-12T20:45:35.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8632025-11-12T20:45:35.695ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8642025-11-12T20:45:35.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8652025-11-12T20:45:35.696ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8662025-11-12T20:45:35.696ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
8672025-11-12T20:45:35.696ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 417.988377ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
8682025-11-12T20:45:35.698ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
8692025-11-12T20:45:35.698ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Dependency
8702025-11-12T20:45:35.698ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
8712025-11-12T20:45:35.698ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 407.357282ms iteration = 1 status = {"errors":[],"probes_by_sled":{}}
8722025-11-12T20:45:35.698ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8732025-11-12T20:45:35.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8742025-11-12T20:45:35.708ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8752025-11-12T20:45:35.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8762025-11-12T20:45:35.715ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8772025-11-12T20:45:35.719ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
8782025-11-12T20:45:35.719ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8792025-11-12T20:45:35.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8802025-11-12T20:45:35.722ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8812025-11-12T20:45:35.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8822025-11-12T20:45:35.724ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8832025-11-12T20:45:35.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
8842025-11-12T20:45:35.727ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8852025-11-12T20:45:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8862025-11-12T20:45:35.731ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8872025-11-12T20:45:35.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8882025-11-12T20:45:35.733ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8892025-11-12T20:45:35.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
8902025-11-12T20:45:35.737ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
8912025-11-12T20:45:35.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8922025-11-12T20:45:35.739ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 460.822666ms iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":0,"put_config_err":0,"put_config_ok":0,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":0,"put_config_err":0,"put_config_ok":0,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[]}
8932025-11-12T20:45:35.743ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 464.660225ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
8942025-11-12T20:45:35.750ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8952025-11-12T20:45:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8962025-11-12T20:45:35.757ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8972025-11-12T20:45:35.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8982025-11-12T20:45:35.764ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8992025-11-12T20:45:35.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9002025-11-12T20:45:35.771ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9012025-11-12T20:45:35.774ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9022025-11-12T20:45:35.774ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9032025-11-12T20:45:35.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9042025-11-12T20:45:35.776ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9052025-11-12T20:45:35.779ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9062025-11-12T20:45:35.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9072025-11-12T20:45:35.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9082025-11-12T20:45:35.780ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9092025-11-12T20:45:35.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9102025-11-12T20:45:35.790ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9112025-11-12T20:45:35.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9122025-11-12T20:45:35.793ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9132025-11-12T20:45:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9142025-11-12T20:45:35.799ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9152025-11-12T20:45:35.800ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 521.521776ms iteration = 1 status = {}
9162025-11-12T20:45:35.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9172025-11-12T20:45:35.803ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9182025-11-12T20:45:35.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9192025-11-12T20:45:35.807ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9202025-11-12T20:45:35.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9212025-11-12T20:45:35.809ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9222025-11-12T20:45:35.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9232025-11-12T20:45:35.811ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9242025-11-12T20:45:35.817ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
9252025-11-12T20:45:35.817ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9262025-11-12T20:45:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9272025-11-12T20:45:35.819ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9282025-11-12T20:45:35.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9292025-11-12T20:45:35.821ZDEBGnexus (CockroachClusterAdminClient): Updated CockroachDB cluster backends added = 5 addresses = [[fd00:1122:3344:101::6]:32222, [fd00:1122:3344:101::3]:32222, [fd00:1122:3344:101::4]:32222, [fd00:1122:3344:101::5]:32222, [fd00:1122:3344:101::7]:32222] removed = 0 total = 5
9302025-11-12T20:45:35.821ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9312025-11-12T20:45:35.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9322025-11-12T20:45:35.823ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9332025-11-12T20:45:35.824ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
9342025-11-12T20:45:35.824ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
9352025-11-12T20:45:35.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
9362025-11-12T20:45:35.825ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b7ef63b-796d-48f3-9c30-44b34a5f2a80", "content-length": "476", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9372025-11-12T20:45:35.830ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
9382025-11-12T20:45:35.830ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
9392025-11-12T20:45:35.830ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9402025-11-12T20:45:35.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9412025-11-12T20:45:35.831ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f89bd6fb-dfb8-4dbb-bfb4-68ae04a8fc21", "content-length": "734", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9422025-11-12T20:45:35.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
9432025-11-12T20:45:35.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc06412a-fc05-4396-852e-474a4e642ff7", "content-length": "111", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9442025-11-12T20:45:35.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
9452025-11-12T20:45:35.838ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e2802f2-d42f-4218-913c-f834ae7eecd3", "content-length": "111", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9462025-11-12T20:45:35.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
9472025-11-12T20:45:35.839ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b29da4d-953d-4fdf-aed3-467c0a59093a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9482025-11-12T20:45:35.839ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
9492025-11-12T20:45:35.840ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd9560c2-0cdc-4499-add7-33b1f23e147b", "content-length": "163", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9502025-11-12T20:45:35.840ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
9512025-11-12T20:45:35.840ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "485a6e19-28f0-4f35-b1c6-3d8f1d224577", "content-length": "166", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9522025-11-12T20:45:35.840ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
9532025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "81ee5fcb-5193-4fd4-90d4-3c95f068ce36", "content-length": "166", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9542025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
9552025-11-12T20:45:35.841ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
9562025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9572025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f75fc04-d117-483f-b404-66f87f0fea0e", "content-length": "702", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9582025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9592025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9602025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3fef7a5-b4ce-4c03-905d-7f1a8a2385ea", "content-length": "727", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9612025-11-12T20:45:35.841ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9622025-11-12T20:45:35.842ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55ceb0be-f692-4611-9bd8-4b7f8b7ac522", "content-length": "729", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9632025-11-12T20:45:35.842ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9642025-11-12T20:45:35.842ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15f22018-92b8-4292-b553-8fabc49ee625", "content-length": "728", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9652025-11-12T20:45:35.842ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
9662025-11-12T20:45:35.842ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aaedb600-852b-419e-bf2a-734f9701f1d6", "content-length": "722", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9672025-11-12T20:45:35.842ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
9682025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9692025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "a467362d-4788-4e92-b311-326de3c74dd3", "content-length": "10", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9702025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
9712025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "63e3f933-f230-4936-ad4a-8419e4f9a4e4", "content-length": "144", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9722025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
9732025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb3c0f73-fe31-420f-a2be-92592dffc821", "content-length": "147", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9742025-11-12T20:45:35.843ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
9752025-11-12T20:45:35.844ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2a8e3e3-3817-47b5-b7e9-10f2c4bce1b3", "content-length": "109", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9762025-11-12T20:45:35.844ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
9772025-11-12T20:45:35.844ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "951d8bd4-142c-4053-ba57-7588b61e6031", "content-length": "109", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9782025-11-12T20:45:35.844ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
9792025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "398daa29-8588-4ea0-a7df-d4f32285ed79", "content-length": "163", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9802025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
9812025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
9822025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9832025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9842025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14fd76d1-2574-4fd1-81e0-03d10a92d53c", "content-length": "163", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9852025-11-12T20:45:35.845ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
9862025-11-12T20:45:35.846ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c48d0a73-013e-42ef-a2e2-d82fece37f41", "content-length": "166", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9872025-11-12T20:45:35.846ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
9882025-11-12T20:45:35.846ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ed974f-c453-441c-b165-adfa783f124d", "content-length": "166", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9892025-11-12T20:45:35.846ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
9902025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd779b65-e2c6-48f5-9332-6f1443ac527a", "content-length": "702", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9912025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
9922025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cdabd83-da48-4a8b-a657-0b9e538bb175", "content-length": "727", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9932025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
9942025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df78977b-d014-4b48-bb3e-5cd3f433265e", "content-length": "729", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9952025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
9962025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0aeceaa9-7845-4be2-abaa-a43a458b9458", "content-length": "728", "date": "Wed, 12 Nov 2025 20:45:35 GMT"} })
9972025-11-12T20:45:35.847ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9982025-11-12T20:45:35.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9992025-11-12T20:45:35.851ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10002025-11-12T20:45:35.853ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
10012025-11-12T20:45:35.853ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10022025-11-12T20:45:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10032025-11-12T20:45:35.854ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
10042025-11-12T20:45:35.854ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
10052025-11-12T20:45:35.854ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
10062025-11-12T20:45:35.855ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10072025-11-12T20:45:35.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
10082025-11-12T20:45:35.858ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10092025-11-12T20:45:35.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10102025-11-12T20:45:35.880ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
10112025-11-12T20:45:35.880ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 601.923416ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
10122025-11-12T20:45:35.883ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 185.825598ms iteration = 2 status = {}
10132025-11-12T20:45:35.952ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
10142025-11-12T20:45:35.952ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
10152025-11-12T20:45:35.952ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
10162025-11-12T20:45:35.952ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
10172025-11-12T20:45:35.952ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
10182025-11-12T20:45:35.952ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
10192025-11-12T20:45:35.952ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
10202025-11-12T20:45:35.952ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
10212025-11-12T20:45:36.021ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
10222025-11-12T20:45:36.021ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
10232025-11-12T20:45:36.021ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
10242025-11-12T20:45:36.021ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 721.000157ms
10252025-11-12T20:45:36.021ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 742.962676ms iteration = 1 status = {}
10262025-11-12T20:45:36.627ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee3ba38f-5b30-4f89-a1cf-b02ec5b2ed73", "content-length": "64", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10272025-11-12T20:45:36.627ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
10282025-11-12T20:45:36.748ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab095b2c-fa5d-4ab9-b8e7-63945ed2b7b4", "content-length": "64", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10292025-11-12T20:45:36.748ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
10302025-11-12T20:45:36.809ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4150da00-08c0-4184-9786-71dd574af4b4", "content-length": "64", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10312025-11-12T20:45:36.809ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
10322025-11-12T20:45:36.811ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab3e53a1-4f34-442a-b772-c35f64f4691c", "content-length": "64", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10332025-11-12T20:45:36.811ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
10342025-11-12T20:45:36.819ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cc5e743-eb91-4095-94ff-1c5afb6e4c97", "content-length": "64", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10352025-11-12T20:45:36.819ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
10362025-11-12T20:45:36.938ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c47fb011-a176-44c3-a976-86d41ff7df26", "content-length": "388510", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10372025-11-12T20:45:36.995ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ff0af9-9548-4811-ba00-6cfe08d3ca9e", "content-length": "350170", "date": "Wed, 12 Nov 2025 20:45:36 GMT"} })
10382025-11-12T20:45:37.065ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9870e5c2-a406-494f-8643-36b8aef2faa7", "content-length": "353272", "date": "Wed, 12 Nov 2025 20:45:37 GMT"} })
10392025-11-12T20:45:37.080ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aacb3cd-40da-4be3-a020-4d675b149c19", "content-length": "347035", "date": "Wed, 12 Nov 2025 20:45:37 GMT"} })
10402025-11-12T20:45:37.082ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dfbe35c-0cb3-458c-b1be-e52fc889a45e", "content-length": "357795", "date": "Wed, 12 Nov 2025 20:45:37 GMT"} })
10412025-11-12T20:45:37.084ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
10422025-11-12T20:45:37.084ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
10432025-11-12T20:45:37.084ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
10442025-11-12T20:45:37.085ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
10452025-11-12T20:45:37.085ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
10462025-11-12T20:45:37.085ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
10472025-11-12T20:45:37.085ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
10482025-11-12T20:45:37.085ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
10492025-11-12T20:45:37.086ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10502025-11-12T20:45:37.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10512025-11-12T20:45:37.088ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10522025-11-12T20:45:37.090ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
10532025-11-12T20:45:37.090ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10542025-11-12T20:45:37.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10552025-11-12T20:45:37.109ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10562025-11-12T20:45:37.109ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
10572025-11-12T20:45:37.111ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10582025-11-12T20:45:37.111ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
10592025-11-12T20:45:37.113ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10602025-11-12T20:45:37.116ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
10612025-11-12T20:45:37.116ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
10622025-11-12T20:45:37.538ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 3dfe864c-3abf-4c5c-b7f1-4ecadb18fbc1 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
10632025-11-12T20:45:37.538ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 3dfe864c-3abf-4c5c-b7f1-4ecadb18fbc1 time_started = 2025-11-12 20:45:35.821767 UTC
10642025-11-12T20:45:37.538ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 2 reason = Dependency
10652025-11-12T20:45:37.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
10662025-11-12T20:45:37.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
10672025-11-12T20:45:37.539ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.260283921s iteration = 1 status = {"collection_id":"3dfe864c-3abf-4c5c-b7f1-4ecadb18fbc1","time_done":"2025-11-12 20:45:37.085352 UTC","time_started":"2025-11-12 20:45:35.821767 UTC"}
10682025-11-12T20:45:37.540ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10692025-11-12T20:45:37.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
10702025-11-12T20:45:37.543ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
10712025-11-12T20:45:37.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
10722025-11-12T20:45:37.547ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
10732025-11-12T20:45:37.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
10742025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 296.213946ms iteration = 2 status = {"Loaded":{"collection_id":"eed561bd-0886-44cc-b52c-adbe9a6b6b19","time_loaded":"2025-11-12T20:45:37.538672001Z","time_started":"2025-11-12T20:45:35.875340Z"}}
10752025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 3 reason = Dependency
10762025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
10772025-11-12T20:45:37.835ZWARNnexus (BackgroundTasks): blueprint planning skipped background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:105 reason = no target blueprint loaded
10782025-11-12T20:45:37.835ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:54 reason = no blueprint
10792025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
10802025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 4.6\u{b5}s iteration = 2 status = {"error":"no blueprint"}
10812025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.981\u{b5}s iteration = 3 status = {"Error":"no target blueprint to use as parent for planning"}
10822025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10832025-11-12T20:45:37.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10842025-11-12T20:45:37.838ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10852025-11-12T20:45:37.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10862025-11-12T20:45:37.840ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10872025-11-12T20:45:37.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10882025-11-12T20:45:37.842ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:82 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
10892025-11-12T20:45:37.842ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 7.233767ms iteration = 2 status = {"error":"rack not yet initialized: 21dec677-5bdd-45c9-88cb-9a589af920a7"}
10902025-11-12T20:45:39.118ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10912025-11-12T20:45:39.119ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
10922025-11-12T20:45:39.120ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
10932025-11-12T20:45:39.120ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
10942025-11-12T20:45:39.121ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10952025-11-12T20:45:39.124ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
10962025-11-12T20:45:39.124ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
10972025-11-12T20:45:40.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 2 reason = Timeout
10982025-11-12T20:45:40.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
10992025-11-12T20:45:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11002025-11-12T20:45:40.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11012025-11-12T20:45:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11022025-11-12T20:45:40.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11032025-11-12T20:45:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11042025-11-12T20:45:40.388ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11052025-11-12T20:45:40.388ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 107.608174ms iteration = 2 status = {"config_updated":false}
11062025-11-12T20:45:41.125ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11072025-11-12T20:45:41.125ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11082025-11-12T20:45:41.127ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11092025-11-12T20:45:41.127ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11102025-11-12T20:45:41.129ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11112025-11-12T20:45:41.132ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
11122025-11-12T20:45:41.132ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
11132025-11-12T20:45:43.131ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11142025-11-12T20:45:43.131ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11152025-11-12T20:45:43.132ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11162025-11-12T20:45:43.132ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11172025-11-12T20:45:43.134ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11182025-11-12T20:45:43.137ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
11192025-11-12T20:45:43.137ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
11202025-11-12T20:45:45.138ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11212025-11-12T20:45:45.138ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11222025-11-12T20:45:45.140ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11232025-11-12T20:45:45.140ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11242025-11-12T20:45:45.141ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11252025-11-12T20:45:45.144ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
11262025-11-12T20:45:45.144ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
11272025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
11282025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
11292025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11302025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11312025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 3 reason = Timeout
11322025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11332025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11342025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11352025-11-12T20:45:45.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11362025-11-12T20:45:45.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11372025-11-12T20:45:45.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11382025-11-12T20:45:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11392025-11-12T20:45:45.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11402025-11-12T20:45:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11412025-11-12T20:45:45.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11422025-11-12T20:45:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11432025-11-12T20:45:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11442025-11-12T20:45:45.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11452025-11-12T20:45:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11462025-11-12T20:45:45.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11472025-11-12T20:45:45.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.798232ms iteration = 3 status = {"config_updated":false}
11482025-11-12T20:45:45.285ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:71
11492025-11-12T20:45:45.285ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 8.246353ms iteration = 2 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
11502025-11-12T20:45:45.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.758256ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11512025-11-12T20:45:47.147ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11522025-11-12T20:45:47.147ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11532025-11-12T20:45:47.148ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11542025-11-12T20:45:47.148ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11552025-11-12T20:45:47.150ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11562025-11-12T20:45:47.153ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
11572025-11-12T20:45:47.153ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
11582025-11-12T20:45:49.155ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11592025-11-12T20:45:49.155ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11602025-11-12T20:45:49.157ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
11612025-11-12T20:45:49.157ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
11622025-11-12T20:45:49.159ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11632025-11-12T20:45:49.162ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
11642025-11-12T20:45:49.162ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
11652025-11-12T20:45:50.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 3 reason = Timeout
11662025-11-12T20:45:50.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
11672025-11-12T20:45:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
11682025-11-12T20:45:50.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 4 reason = Timeout
11692025-11-12T20:45:50.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11702025-11-12T20:45:50.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11712025-11-12T20:45:50.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11722025-11-12T20:45:50.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11732025-11-12T20:45:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
11742025-11-12T20:45:50.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
11752025-11-12T20:45:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
11762025-11-12T20:45:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
11772025-11-12T20:45:50.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
11782025-11-12T20:45:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
11792025-11-12T20:45:50.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 2 reason = Timeout
11802025-11-12T20:45:50.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
11812025-11-12T20:45:50.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
11822025-11-12T20:45:50.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.989556ms iteration = 4 status = {"config_updated":false}
11832025-11-12T20:45:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
11842025-11-12T20:45:50.287ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = eed561bd-0886-44cc-b52c-adbe9a6b6b19 original_time_started = 2025-11-12 20:45:35.875340 UTC
11852025-11-12T20:45:50.287ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.986913ms iteration = 3 status = {"Loaded":{"collection_id":"eed561bd-0886-44cc-b52c-adbe9a6b6b19","time_loaded":"2025-11-12T20:45:50.278244963Z","time_started":"2025-11-12T20:45:35.875340Z"}}
11862025-11-12T20:45:50.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11872025-11-12T20:45:50.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
11882025-11-12T20:45:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
11892025-11-12T20:45:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
11902025-11-12T20:45:50.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 2 reason = Timeout
11912025-11-12T20:45:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
11922025-11-12T20:45:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
11932025-11-12T20:45:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
11942025-11-12T20:45:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
11952025-11-12T20:45:50.292ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.303013ms iteration = 2 status = "NoSitrep"
11962025-11-12T20:45:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972025-11-12T20:45:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
11982025-11-12T20:45:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
11992025-11-12T20:45:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
12002025-11-12T20:45:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
12012025-11-12T20:45:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
12022025-11-12T20:45:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12032025-11-12T20:45:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
12042025-11-12T20:45:50.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
12052025-11-12T20:45:50.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.405614ms iteration = 2 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
12062025-11-12T20:45:51.164ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12072025-11-12T20:45:51.164ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12082025-11-12T20:45:51.166ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12092025-11-12T20:45:51.166ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12102025-11-12T20:45:51.167ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12112025-11-12T20:45:51.170ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12122025-11-12T20:45:51.170ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12132025-11-12T20:45:53.171ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12142025-11-12T20:45:53.172ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12152025-11-12T20:45:53.174ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12162025-11-12T20:45:53.174ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12172025-11-12T20:45:53.175ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12182025-11-12T20:45:53.178ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12192025-11-12T20:45:53.178ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12202025-11-12T20:45:55.178ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12212025-11-12T20:45:55.179ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12222025-11-12T20:45:55.180ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12232025-11-12T20:45:55.180ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12242025-11-12T20:45:55.182ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12252025-11-12T20:45:55.185ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12262025-11-12T20:45:55.185ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12272025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
12282025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12292025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
12302025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12312025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 5 reason = Timeout
12322025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12332025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12342025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12352025-11-12T20:45:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12362025-11-12T20:45:55.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12372025-11-12T20:45:55.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12382025-11-12T20:45:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12392025-11-12T20:45:55.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12402025-11-12T20:45:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12412025-11-12T20:45:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
12422025-11-12T20:45:55.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12432025-11-12T20:45:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12442025-11-12T20:45:55.287ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:71
12452025-11-12T20:45:55.287ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 7.785731ms iteration = 3 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
12462025-11-12T20:45:55.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12472025-11-12T20:45:55.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12482025-11-12T20:45:55.320ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
12492025-11-12T20:45:55.320ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 40.804687ms iteration = 5 status = {"config_updated":false}
12502025-11-12T20:45:55.353ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.039096ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12512025-11-12T20:45:57.184ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12522025-11-12T20:45:57.185ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12532025-11-12T20:45:57.186ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12542025-11-12T20:45:57.187ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12552025-11-12T20:45:57.188ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12562025-11-12T20:45:57.191ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12572025-11-12T20:45:57.191ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12582025-11-12T20:45:59.191ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12592025-11-12T20:45:59.191ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12602025-11-12T20:45:59.193ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12612025-11-12T20:45:59.193ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12622025-11-12T20:45:59.195ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12632025-11-12T20:45:59.198ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12642025-11-12T20:45:59.198ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12652025-11-12T20:46:00.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 6 reason = Timeout
12662025-11-12T20:46:00.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12672025-11-12T20:46:00.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12682025-11-12T20:46:00.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12692025-11-12T20:46:00.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
12702025-11-12T20:46:00.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12712025-11-12T20:46:00.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
12722025-11-12T20:46:00.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
12732025-11-12T20:46:00.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.338874ms iteration = 6 status = {"config_updated":false}
12742025-11-12T20:46:01.200ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12752025-11-12T20:46:01.200ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12762025-11-12T20:46:01.202ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12772025-11-12T20:46:01.202ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12782025-11-12T20:46:01.204ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12792025-11-12T20:46:01.207ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12802025-11-12T20:46:01.207ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12812025-11-12T20:46:03.209ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12822025-11-12T20:46:03.210ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12832025-11-12T20:46:03.211ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12842025-11-12T20:46:03.211ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12852025-11-12T20:46:03.212ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12862025-11-12T20:46:03.215ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12872025-11-12T20:46:03.215ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12882025-11-12T20:46:05.216ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12892025-11-12T20:46:05.216ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12902025-11-12T20:46:05.218ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
12912025-11-12T20:46:05.218ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
12922025-11-12T20:46:05.220ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12932025-11-12T20:46:05.223ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
12942025-11-12T20:46:05.223ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: 21dec677-5bdd-45c9-88cb-9a589af920a7, time_created: 2025-11-12T20:45:34.688537Z, time_modified: 2025-11-12T20:45:34.688537Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:779
12952025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
12962025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
12972025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12982025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 2 reason = Timeout
12992025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
13002025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
13012025-11-12T20:46:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
13022025-11-12T20:46:05.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 4 reason = Timeout
13032025-11-12T20:46:05.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
13042025-11-12T20:46:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
13052025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
13062025-11-12T20:46:05.277ZERROnexus (BackgroundTasks): nexus is running but no service zone nat values could be generated from inventory background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:199
13072025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 15\u{b5}s iteration = 2 status = {"error":"nexus is running but no service zone nat values could be generated from inventory"}
13082025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
13092025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 2 reason = Timeout
13102025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
13112025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13122025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13132025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13142025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
13152025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
13162025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13172025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
13182025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13192025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 7 reason = Timeout
13202025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13212025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
13222025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
13232025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
13242025-11-12T20:46:05.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
13252025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13262025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
13272025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13282025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
13292025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
13302025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13312025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
13322025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
13332025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13342025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
13352025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13362025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
13372025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13382025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 2 reason = Timeout
13392025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
13402025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
13412025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13422025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 2 reason = Timeout
13432025-11-12T20:46:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13442025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13452025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
13462025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.877544ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
13472025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
13482025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
13492025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13502025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13512025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13522025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13532025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13542025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
13552025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
13562025-11-12T20:46:05.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 960, tv_nsec: 775187248 } })
13572025-11-12T20:46:05.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 960, tv_nsec: 775213758 } })
13582025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13592025-11-12T20:46:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13602025-11-12T20:46:05.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 960, tv_nsec: 775723802 } })
13612025-11-12T20:46:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13622025-11-12T20:46:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
13632025-11-12T20:46:05.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
13642025-11-12T20:46:05.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13652025-11-12T20:46:05.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13662025-11-12T20:46:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13672025-11-12T20:46:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13682025-11-12T20:46:05.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13692025-11-12T20:46:05.282ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13702025-11-12T20:46:05.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13712025-11-12T20:46:05.282ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13722025-11-12T20:46:05.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13732025-11-12T20:46:05.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742025-11-12T20:46:05.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 3 reason = Timeout
13752025-11-12T20:46:05.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
13762025-11-12T20:46:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
13772025-11-12T20:46:05.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13782025-11-12T20:46:05.287ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13792025-11-12T20:46:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
13802025-11-12T20:46:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
13812025-11-12T20:46:05.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13822025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13832025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
13842025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
13852025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
13862025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13872025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
13882025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13892025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13902025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13912025-11-12T20:46:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13922025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13932025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13942025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13952025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13962025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
13972025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13982025-11-12T20:46:05.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13992025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
14002025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14012025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14022025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14032025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14042025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 3 reason = Timeout
14052025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
14062025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14072025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
14082025-11-12T20:46:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
14092025-11-12T20:46:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
14102025-11-12T20:46:05.291ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 13.393056ms iteration = 2 status = {}
14112025-11-12T20:46:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
14122025-11-12T20:46:05.291ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:82 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
14132025-11-12T20:46:05.291ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 13.772747ms iteration = 3 status = {"error":"rack not yet initialized: 21dec677-5bdd-45c9-88cb-9a589af920a7"}
14142025-11-12T20:46:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
14152025-11-12T20:46:05.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
14162025-11-12T20:46:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
14172025-11-12T20:46:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
14182025-11-12T20:46:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.834544ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14192025-11-12T20:46:05.292ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
14202025-11-12T20:46:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 15.223924ms iteration = 7 status = {"config_updated":false}
14212025-11-12T20:46:05.293ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14222025-11-12T20:46:05.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14232025-11-12T20:46:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14242025-11-12T20:46:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14252025-11-12T20:46:05.295ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.564593ms iteration = 3 status = "NoSitrep"
14262025-11-12T20:46:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
14272025-11-12T20:46:05.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
14282025-11-12T20:46:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
14292025-11-12T20:46:05.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
14302025-11-12T20:46:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
14312025-11-12T20:46:05.309ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14322025-11-12T20:46:05.312ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
14332025-11-12T20:46:05.313ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
14342025-11-12T20:46:05.313ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 22.732118ms iteration = 3 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
14352025-11-12T20:46:05.329ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
14362025-11-12T20:46:05.329ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 51.574795ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
14372025-11-12T20:46:05.357ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14382025-11-12T20:46:05.357ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
14392025-11-12T20:46:05.357ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14402025-11-12T20:46:05.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
14412025-11-12T20:46:05.365ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14422025-11-12T20:46:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14432025-11-12T20:46:05.366ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14442025-11-12T20:46:05.366ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
14452025-11-12T20:46:05.366ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
14462025-11-12T20:46:05.366ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.423488ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
14472025-11-12T20:46:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
14482025-11-12T20:46:05.368ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14492025-11-12T20:46:05.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 93.097799ms iteration = 2 status = {"errors":[],"finish_invoked_ok":[]}
14502025-11-12T20:46:05.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
14512025-11-12T20:46:05.372ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14522025-11-12T20:46:05.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14532025-11-12T20:46:05.373ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 96.185059ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
14542025-11-12T20:46:05.380ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14552025-11-12T20:46:05.381ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
14562025-11-12T20:46:05.381ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14572025-11-12T20:46:05.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14582025-11-12T20:46:05.381ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a3fc776-95d6-4c2f-92e3-a90ed5e70b58", "content-length": "26", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
14592025-11-12T20:46:05.381ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14602025-11-12T20:46:05.382ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
14612025-11-12T20:46:05.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
14622025-11-12T20:46:05.383ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14632025-11-12T20:46:05.384ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
14642025-11-12T20:46:05.384ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "956b3070-2ba1-44d0-97dc-06f3c7aca2b5", "content-length": "78", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
14652025-11-12T20:46:05.384ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
14662025-11-12T20:46:05.384ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
14672025-11-12T20:46:05.384ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
14682025-11-12T20:46:05.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
14692025-11-12T20:46:05.385ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 107.373889ms iteration = 2 status = {"errors":[],"requests_created_ok":[]}
14702025-11-12T20:46:05.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
14712025-11-12T20:46:05.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
14722025-11-12T20:46:05.386ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
14732025-11-12T20:46:05.387ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
14742025-11-12T20:46:05.387ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
14752025-11-12T20:46:05.387ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 112.553763ms iteration = 2 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
14762025-11-12T20:46:05.388ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14772025-11-12T20:46:05.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
14782025-11-12T20:46:05.391ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
14792025-11-12T20:46:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
14802025-11-12T20:46:05.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
14812025-11-12T20:46:05.391ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
14822025-11-12T20:46:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
14832025-11-12T20:46:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
14842025-11-12T20:46:05.428ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 152.704902ms iteration = 2 status = {}
14852025-11-12T20:46:05.433ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
14862025-11-12T20:46:05.433ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 155.813603ms iteration = 2 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
14872025-11-12T20:46:05.435ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14882025-11-12T20:46:05.435ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
14892025-11-12T20:46:05.435ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 158.150792ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
14902025-11-12T20:46:05.438ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 160.918925ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
14912025-11-12T20:46:05.440ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
14922025-11-12T20:46:05.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
14932025-11-12T20:46:05.442ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
14942025-11-12T20:46:05.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
14952025-11-12T20:46:05.445ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
14962025-11-12T20:46:05.445ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
14972025-11-12T20:46:05.446ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 168.536031ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
14982025-11-12T20:46:05.446ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ac8366a-a905-40dc-b851-b688e6e1762c", "content-length": "81", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
14992025-11-12T20:46:05.450ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
15002025-11-12T20:46:05.450ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
15012025-11-12T20:46:05.450ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "34d49e1a-d32f-446c-b2d5-675f85ac266d", "content-length": "81", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
15022025-11-12T20:46:05.492ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
15032025-11-12T20:46:05.492ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
15042025-11-12T20:46:05.492ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d5fde91-7fba-4126-aeb2-a0a2bc35c09b", "content-length": "81", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
15052025-11-12T20:46:05.493ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 216.0042ms iteration = 2 status = {"disabled":false,"errors":[],"sps":[]}
15062025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 441.125606ms iteration = 4 status = {"Loaded":{"collection_id":"3ea6c7e4-601e-4623-bf5d-213cbc0f74e1","time_loaded":"2025-11-12T20:46:05.276527411Z","time_started":"2025-11-12T20:46:01.047491Z"}}
15072025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 4 reason = Dependency
15082025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Dependency
15092025-11-12T20:46:05.717ZWARNnexus (BackgroundTasks): blueprint planning skipped background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:105 reason = no target blueprint loaded
15102025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 3 reason = Dependency
15112025-11-12T20:46:05.717ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:54 reason = no blueprint
15122025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 6.401\u{b5}s iteration = 3 status = {"error":"no blueprint"}
15132025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 17.02\u{b5}s iteration = 4 status = {"Error":"no target blueprint to use as parent for planning"}
15142025-11-12T20:46:05.717ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15152025-11-12T20:46:05.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15162025-11-12T20:46:05.719ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15172025-11-12T20:46:05.722ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15182025-11-12T20:46:05.722ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15192025-11-12T20:46:05.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15202025-11-12T20:46:05.724ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:82 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
15212025-11-12T20:46:05.724ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 6.412716ms iteration = 4 status = {"error":"rack not yet initialized: 21dec677-5bdd-45c9-88cb-9a589af920a7"}
15222025-11-12T20:46:05.775ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:71
15232025-11-12T20:46:05.775ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 498.064459ms iteration = 4 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
15242025-11-12T20:46:05.783ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15252025-11-12T20:46:05.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15262025-11-12T20:46:05.809ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
15272025-11-12T20:46:05.809ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
15282025-11-12T20:46:05.810ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f47464-8946-48ad-807c-47f798a762f6", "content-length": "312", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
15292025-11-12T20:46:05.810ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15302025-11-12T20:46:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15312025-11-12T20:46:05.827ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15322025-11-12T20:46:05.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15332025-11-12T20:46:05.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15342025-11-12T20:46:05.832ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
15352025-11-12T20:46:05.832ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15362025-11-12T20:46:05.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15372025-11-12T20:46:05.833ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15382025-11-12T20:46:05.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
15392025-11-12T20:46:05.835ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15402025-11-12T20:46:05.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15412025-11-12T20:46:05.870ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15422025-11-12T20:46:05.871ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f18a029-47e2-4097-90a2-a718ddb064e4", "content-length": "691", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
15432025-11-12T20:46:05.873ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
15442025-11-12T20:46:05.873ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
15452025-11-12T20:46:05.873ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
15462025-11-12T20:46:05.876ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 599.009211ms iteration = 2 status = {}
15472025-11-12T20:46:05.887ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
15482025-11-12T20:46:05.887ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
15492025-11-12T20:46:05.887ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "02a51228-30e2-4065-aec1-ed813319e1fb", "date": "Wed, 12 Nov 2025 20:46:05 GMT"} })
15502025-11-12T20:46:05.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15512025-11-12T20:46:05.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15522025-11-12T20:46:05.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15532025-11-12T20:46:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15542025-11-12T20:46:05.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15552025-11-12T20:46:05.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15562025-11-12T20:46:05.902ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15572025-11-12T20:46:05.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15582025-11-12T20:46:05.904ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15592025-11-12T20:46:05.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15602025-11-12T20:46:05.906ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15612025-11-12T20:46:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15622025-11-12T20:46:05.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15632025-11-12T20:46:05.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15642025-11-12T20:46:05.910ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15652025-11-12T20:46:05.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15662025-11-12T20:46:05.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15672025-11-12T20:46:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15682025-11-12T20:46:05.913ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15692025-11-12T20:46:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15702025-11-12T20:46:05.915ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15712025-11-12T20:46:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
15722025-11-12T20:46:05.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15732025-11-12T20:46:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15742025-11-12T20:46:05.921ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15752025-11-12T20:46:05.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15762025-11-12T20:46:05.922ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15772025-11-12T20:46:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15782025-11-12T20:46:05.924ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15792025-11-12T20:46:05.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
15802025-11-12T20:46:05.928ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15812025-11-12T20:46:05.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15822025-11-12T20:46:05.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15832025-11-12T20:46:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15842025-11-12T20:46:05.936ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15852025-11-12T20:46:05.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15862025-11-12T20:46:05.937ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15872025-11-12T20:46:05.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15882025-11-12T20:46:05.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15892025-11-12T20:46:05.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15902025-11-12T20:46:05.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15912025-11-12T20:46:05.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15922025-11-12T20:46:05.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15932025-11-12T20:46:05.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15942025-11-12T20:46:05.944ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15952025-11-12T20:46:05.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15962025-11-12T20:46:05.945ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15972025-11-12T20:46:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ById(001de000-c470-4000-8000-000000000001) } result = Ok(())
15982025-11-12T20:46:05.950ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15992025-11-12T20:46:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16002025-11-12T20:46:05.951ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16012025-11-12T20:46:05.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16022025-11-12T20:46:05.953ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16032025-11-12T20:46:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16042025-11-12T20:46:05.954ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16052025-11-12T20:46:05.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16062025-11-12T20:46:05.956ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16072025-11-12T20:46:05.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16082025-11-12T20:46:05.957ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16092025-11-12T20:46:05.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16102025-11-12T20:46:05.959ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16112025-11-12T20:46:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16122025-11-12T20:46:05.960ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16132025-11-12T20:46:05.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ById(001de000-c470-4000-8000-000000000002) } result = Ok(())
16142025-11-12T20:46:05.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16152025-11-12T20:46:05.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16162025-11-12T20:46:05.967ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16172025-11-12T20:46:05.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16182025-11-12T20:46:05.968ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16192025-11-12T20:46:05.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16202025-11-12T20:46:05.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16212025-11-12T20:46:05.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16222025-11-12T20:46:05.971ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16232025-11-12T20:46:05.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16242025-11-12T20:46:05.972ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16252025-11-12T20:46:05.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16262025-11-12T20:46:05.974ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16272025-11-12T20:46:05.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16282025-11-12T20:46:05.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16292025-11-12T20:46:05.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ById(001de000-c470-4000-8000-000000000003) } result = Ok(())
16302025-11-12T20:46:05.981ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16312025-11-12T20:46:05.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16322025-11-12T20:46:05.983ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16332025-11-12T20:46:05.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16342025-11-12T20:46:05.984ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16352025-11-12T20:46:05.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16362025-11-12T20:46:05.985ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16372025-11-12T20:46:05.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16382025-11-12T20:46:05.988ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16392025-11-12T20:46:05.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16402025-11-12T20:46:05.990ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16412025-11-12T20:46:05.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16422025-11-12T20:46:05.991ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16432025-11-12T20:46:05.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16442025-11-12T20:46:05.993ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16452025-11-12T20:46:06.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16462025-11-12T20:46:06.000ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16472025-11-12T20:46:06.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16482025-11-12T20:46:06.002ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16492025-11-12T20:46:06.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16502025-11-12T20:46:06.004ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16512025-11-12T20:46:06.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16522025-11-12T20:46:06.007ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16532025-11-12T20:46:06.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16542025-11-12T20:46:06.009ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16552025-11-12T20:46:06.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16562025-11-12T20:46:06.010ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16572025-11-12T20:46:06.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16582025-11-12T20:46:06.012ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16592025-11-12T20:46:06.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16602025-11-12T20:46:06.014ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16612025-11-12T20:46:06.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
16622025-11-12T20:46:06.019ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16632025-11-12T20:46:06.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16642025-11-12T20:46:06.021ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16652025-11-12T20:46:06.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16662025-11-12T20:46:06.023ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16672025-11-12T20:46:06.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16682025-11-12T20:46:06.024ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16692025-11-12T20:46:06.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16702025-11-12T20:46:06.027ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16712025-11-12T20:46:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16722025-11-12T20:46:06.028ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16732025-11-12T20:46:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16742025-11-12T20:46:06.030ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16752025-11-12T20:46:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16762025-11-12T20:46:06.036ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16772025-11-12T20:46:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-c470-4000-8000-000000000003, lookup_type: ByName("boundary-ntp") } result = Ok(())
16782025-11-12T20:46:06.041ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16792025-11-12T20:46:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16802025-11-12T20:46:06.043ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16812025-11-12T20:46:06.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16822025-11-12T20:46:06.044ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16832025-11-12T20:46:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16842025-11-12T20:46:06.047ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16852025-11-12T20:46:06.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16862025-11-12T20:46:06.048ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16872025-11-12T20:46:06.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16882025-11-12T20:46:06.051ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16892025-11-12T20:46:06.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16902025-11-12T20:46:06.053ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16912025-11-12T20:46:06.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16922025-11-12T20:46:06.055ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16932025-11-12T20:46:06.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000002, lookup_type: ByName("default") } result = Ok(())
16942025-11-12T20:46:06.061ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16952025-11-12T20:46:06.061ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "78de76f5-338a-4c33-8dc4-dee399c89c3e", "date": "Wed, 12 Nov 2025 20:46:06 GMT"} })
16962025-11-12T20:46:06.063ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 785.415936ms iteration = 3 status = {}
16972025-11-12T20:46:07.225ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
16982025-11-12T20:46:07.225ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
16992025-11-12T20:46:07.227ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
17002025-11-12T20:46:07.227ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Database result = Ok(())
17012025-11-12T20:46:07.229ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022025-11-12T20:46:07.232ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3 (built_in_user), .. }) resource = Rack { parent: Fleet, key: 21dec677-5bdd-45c9-88cb-9a589af920a7, lookup_type: ById(21dec677-5bdd-45c9-88cb-9a589af920a7) } result = Ok(())
17032025-11-12T20:46:07.232ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:776
17042025-11-12T20:46:07.232ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17052025-11-12T20:46:07.232ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17062025-11-12T20:46:07.236ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17072025-11-12T20:46:07.237ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17082025-11-12T20:46:07.238ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17092025-11-12T20:46:07.238ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17102025-11-12T20:46:07.240ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17112025-11-12T20:46:07.240ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17122025-11-12T20:46:07.242ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17132025-11-12T20:46:07.242ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17142025-11-12T20:46:07.243ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17152025-11-12T20:46:07.244ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17162025-11-12T20:46:07.245ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17172025-11-12T20:46:07.249ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17182025-11-12T20:46:07.249ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17192025-11-12T20:46:07.249ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17202025-11-12T20:46:07.250ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17212025-11-12T20:46:07.250ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17222025-11-12T20:46:07.252ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17232025-11-12T20:46:07.252ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17242025-11-12T20:46:07.253ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17252025-11-12T20:46:07.257ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17262025-11-12T20:46:07.257ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17272025-11-12T20:46:07.257ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17282025-11-12T20:46:07.262ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17292025-11-12T20:46:07.262ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17302025-11-12T20:46:07.263ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17312025-11-12T20:46:07.263ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17322025-11-12T20:46:07.264ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17332025-11-12T20:46:07.265ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17342025-11-12T20:46:07.266ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17352025-11-12T20:46:07.266ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17362025-11-12T20:46:07.267ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17372025-11-12T20:46:07.267ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17382025-11-12T20:46:07.269ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17392025-11-12T20:46:07.269ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17402025-11-12T20:46:07.270ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17412025-11-12T20:46:07.274ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17422025-11-12T20:46:07.274ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17432025-11-12T20:46:07.274ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17442025-11-12T20:46:07.275ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17452025-11-12T20:46:07.275ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17462025-11-12T20:46:07.277ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17472025-11-12T20:46:07.277ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17482025-11-12T20:46:07.279ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17492025-11-12T20:46:07.279ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17502025-11-12T20:46:07.282ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17512025-11-12T20:46:07.282ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17522025-11-12T20:46:07.285ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17532025-11-12T20:46:07.285ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17542025-11-12T20:46:07.286ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17552025-11-12T20:46:07.286ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17562025-11-12T20:46:07.287ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17572025-11-12T20:46:07.292ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
17582025-11-12T20:46:07.292ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17592025-11-12T20:46:07.292ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17602025-11-12T20:46:07.293ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17612025-11-12T20:46:07.294ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17622025-11-12T20:46:07.295ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17632025-11-12T20:46:07.295ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17642025-11-12T20:46:07.297ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17652025-11-12T20:46:07.301ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
17662025-11-12T20:46:07.301ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17672025-11-12T20:46:07.301ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17682025-11-12T20:46:07.308ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17692025-11-12T20:46:07.308ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17702025-11-12T20:46:07.309ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17712025-11-12T20:46:07.309ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17722025-11-12T20:46:07.311ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17732025-11-12T20:46:07.311ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17742025-11-12T20:46:07.313ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17752025-11-12T20:46:07.313ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17762025-11-12T20:46:07.315ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17772025-11-12T20:46:07.315ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17782025-11-12T20:46:07.316ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17792025-11-12T20:46:07.316ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17802025-11-12T20:46:07.318ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17812025-11-12T20:46:07.318ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17822025-11-12T20:46:07.319ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17832025-11-12T20:46:07.319ZINFOnexus (dropshot_lockstep): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12232 remote_addr = [fd00:1122:3344:101::2]:44574
17842025-11-12T20:46:07.324ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17852025-11-12T20:46:07.324ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17862025-11-12T20:46:07.324ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17872025-11-12T20:46:07.326ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17882025-11-12T20:46:07.326ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17892025-11-12T20:46:07.328ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17902025-11-12T20:46:07.328ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17912025-11-12T20:46:07.330ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17922025-11-12T20:46:07.334ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17932025-11-12T20:46:07.334ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17942025-11-12T20:46:07.334ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17952025-11-12T20:46:07.341ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
17962025-11-12T20:46:07.341ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
17972025-11-12T20:46:07.342ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
17982025-11-12T20:46:07.343ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17992025-11-12T20:46:07.345ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Fleet result = Ok(())
18002025-11-12T20:46:07.345ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
18012025-11-12T20:46:07.345ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
18022025-11-12T20:46:07.347ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
18032025-11-12T20:46:07.348ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
18042025-11-12T20:46:07.348ZDEBGnexus (InternalApi): resolved 3 rules for sleds
18052025-11-12T20:46:07.409ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
18062025-11-12T20:46:07.409ZDEBGnexus (InternalApi): sending firewall rules to sled agents
18072025-11-12T20:46:07.440ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
18082025-11-12T20:46:07.440ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
18092025-11-12T20:46:07.443ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
18102025-11-12T20:46:07.443ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Database result = Ok(())
18112025-11-12T20:46:07.445ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18122025-11-12T20:46:07.448ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
18132025-11-12T20:46:07.448ZDEBGnexus (InternalApi): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18142025-11-12T20:46:07.448ZDEBGnexus (InternalApi): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9e73f0c9-5845-4f36-8ac6-3f1a21101ca6", "date": "Wed, 12 Nov 2025 20:46:07 GMT"} })
18152025-11-12T20:46:07.449ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:491
18162025-11-12T20:46:07.449ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
18172025-11-12T20:46:10.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 8 reason = Timeout
18182025-11-12T20:46:10.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18192025-11-12T20:46:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18202025-11-12T20:46:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18212025-11-12T20:46:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18222025-11-12T20:46:10.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18232025-11-12T20:46:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18242025-11-12T20:46:10.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
18252025-11-12T20:46:10.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.093454ms iteration = 8 status = {"config_updated":false}
18262025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
18272025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18282025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
18292025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 9 reason = Timeout
18302025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18312025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18322025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18332025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18342025-11-12T20:46:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18352025-11-12T20:46:15.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18362025-11-12T20:46:15.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18372025-11-12T20:46:15.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18382025-11-12T20:46:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18392025-11-12T20:46:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18402025-11-12T20:46:15.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18412025-11-12T20:46:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18422025-11-12T20:46:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18432025-11-12T20:46:15.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18442025-11-12T20:46:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18452025-11-12T20:46:15.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18462025-11-12T20:46:15.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
18472025-11-12T20:46:15.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.944715ms iteration = 9 status = {"config_updated":false}
18482025-11-12T20:46:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18492025-11-12T20:46:15.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18502025-11-12T20:46:15.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
18512025-11-12T20:46:15.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18522025-11-12T20:46:15.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18532025-11-12T20:46:15.318ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:90 target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
18542025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.99216ms iteration = 5 status = {"enabled":true,"status":"first target blueprint","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z","time_found":"2025-11-12T20:46:15.318412203Z"}
18552025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
18562025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
18572025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 5 reason = Dependency
18582025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18592025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18602025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18612025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18622025-11-12T20:46:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18632025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18642025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18652025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18662025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18672025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18682025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18692025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18702025-11-12T20:46:15.319ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
18712025-11-12T20:46:15.319ZINFOnexus (NexusQuiesceHandle): initial state file = nexus/src/app/quiesce.rs:108 state = Running
18722025-11-12T20:46:15.319ZINFOnexus (SagaQuiesceHandle): initial quiesce state file = nexus/types/src/quiesce.rs:230 initial_state = Allowed
18732025-11-12T20:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18742025-11-12T20:46:15.320ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18752025-11-12T20:46:15.322ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
18762025-11-12T20:46:15.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
18772025-11-12T20:46:15.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18782025-11-12T20:46:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18792025-11-12T20:46:15.324ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18802025-11-12T20:46:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18812025-11-12T20:46:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18822025-11-12T20:46:15.326ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18832025-11-12T20:46:15.326ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = ExternalNetworking step id = Ensure
18842025-11-12T20:46:15.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18852025-11-12T20:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18862025-11-12T20:46:15.329ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18872025-11-12T20:46:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18882025-11-12T20:46:15.329ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18892025-11-12T20:46:15.329ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 52.050457ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18902025-11-12T20:46:15.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
18912025-11-12T20:46:15.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
18922025-11-12T20:46:15.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18932025-11-12T20:46:15.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18942025-11-12T20:46:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18952025-11-12T20:46:15.332ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18962025-11-12T20:46:15.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18972025-11-12T20:46:15.332ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18982025-11-12T20:46:15.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18992025-11-12T20:46:15.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19002025-11-12T20:46:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19012025-11-12T20:46:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
19022025-11-12T20:46:15.334ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19032025-11-12T20:46:15.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19042025-11-12T20:46:15.348ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19052025-11-12T20:46:15.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062025-11-12T20:46:15.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
19072025-11-12T20:46:15.351ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
19082025-11-12T20:46:15.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
19092025-11-12T20:46:15.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
19102025-11-12T20:46:15.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19112025-11-12T20:46:15.353ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 35.078517ms iteration = 2 status = {"nsuccess":5}
19122025-11-12T20:46:15.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19132025-11-12T20:46:15.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19142025-11-12T20:46:15.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19152025-11-12T20:46:15.357ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
19162025-11-12T20:46:15.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19172025-11-12T20:46:15.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19182025-11-12T20:46:15.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19192025-11-12T20:46:15.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19202025-11-12T20:46:15.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19212025-11-12T20:46:15.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19222025-11-12T20:46:15.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19232025-11-12T20:46:15.376ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
19242025-11-12T20:46:15.379ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
19252025-11-12T20:46:15.380ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
19262025-11-12T20:46:15.382ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
19272025-11-12T20:46:15.383ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
19282025-11-12T20:46:15.385ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
19292025-11-12T20:46:15.386ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
19302025-11-12T20:46:15.387ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
19312025-11-12T20:46:15.389ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
19322025-11-12T20:46:15.390ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
19332025-11-12T20:46:15.391ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
19342025-11-12T20:46:15.393ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
19352025-11-12T20:46:15.393ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = SledList step id = Fetch
19362025-11-12T20:46:15.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19372025-11-12T20:46:15.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19382025-11-12T20:46:15.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19392025-11-12T20:46:15.398ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
19402025-11-12T20:46:15.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19412025-11-12T20:46:15.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19422025-11-12T20:46:15.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19432025-11-12T20:46:15.401ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
19442025-11-12T20:46:15.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19452025-11-12T20:46:15.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19462025-11-12T20:46:15.404ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = DeployNexusRecords step id = Ensure
19472025-11-12T20:46:15.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19482025-11-12T20:46:15.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19492025-11-12T20:46:15.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19502025-11-12T20:46:15.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19512025-11-12T20:46:15.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19522025-11-12T20:46:15.407ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
19532025-11-12T20:46:15.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19542025-11-12T20:46:15.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19552025-11-12T20:46:15.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19562025-11-12T20:46:15.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19572025-11-12T20:46:15.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19582025-11-12T20:46:15.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19592025-11-12T20:46:15.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19602025-11-12T20:46:15.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19612025-11-12T20:46:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19622025-11-12T20:46:15.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
19632025-11-12T20:46:15.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19642025-11-12T20:46:15.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19652025-11-12T20:46:15.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19662025-11-12T20:46:15.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19672025-11-12T20:46:15.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19682025-11-12T20:46:15.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
19692025-11-12T20:46:15.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19702025-11-12T20:46:15.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19712025-11-12T20:46:15.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19722025-11-12T20:46:15.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
19732025-11-12T20:46:15.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19742025-11-12T20:46:15.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19752025-11-12T20:46:15.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19762025-11-12T20:46:15.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19772025-11-12T20:46:15.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19782025-11-12T20:46:15.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
19792025-11-12T20:46:15.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19802025-11-12T20:46:15.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19812025-11-12T20:46:15.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19822025-11-12T20:46:15.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19832025-11-12T20:46:15.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19842025-11-12T20:46:15.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
19852025-11-12T20:46:15.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19862025-11-12T20:46:15.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19872025-11-12T20:46:15.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19882025-11-12T20:46:15.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19892025-11-12T20:46:15.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19902025-11-12T20:46:15.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
19912025-11-12T20:46:15.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19922025-11-12T20:46:15.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19932025-11-12T20:46:15.455ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = SledAgent step id = Ensure
19942025-11-12T20:46:15.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19952025-11-12T20:46:15.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19962025-11-12T20:46:15.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19972025-11-12T20:46:15.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19982025-11-12T20:46:15.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19992025-11-12T20:46:15.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20002025-11-12T20:46:15.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20012025-11-12T20:46:15.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20022025-11-12T20:46:15.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20032025-11-12T20:46:15.478ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20042025-11-12T20:46:15.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20052025-11-12T20:46:15.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20062025-11-12T20:46:15.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20072025-11-12T20:46:15.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20082025-11-12T20:46:15.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20092025-11-12T20:46:15.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20102025-11-12T20:46:15.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20112025-11-12T20:46:15.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20122025-11-12T20:46:15.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20132025-11-12T20:46:15.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20142025-11-12T20:46:15.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20152025-11-12T20:46:15.492ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20162025-11-12T20:46:15.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20172025-11-12T20:46:15.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20182025-11-12T20:46:15.493ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
20192025-11-12T20:46:15.494ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "4bc09768-4432-4d07-97e7-819499389e78", "date": "Wed, 12 Nov 2025 20:46:15 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
20202025-11-12T20:46:15.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20212025-11-12T20:46:15.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20222025-11-12T20:46:15.495ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = FirewallRules step id = Ensure
20232025-11-12T20:46:15.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20242025-11-12T20:46:15.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20252025-11-12T20:46:15.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20262025-11-12T20:46:15.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20272025-11-12T20:46:15.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20282025-11-12T20:46:15.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20292025-11-12T20:46:15.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20302025-11-12T20:46:15.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20312025-11-12T20:46:15.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20322025-11-12T20:46:15.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
20332025-11-12T20:46:15.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20342025-11-12T20:46:15.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20352025-11-12T20:46:15.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20362025-11-12T20:46:15.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20372025-11-12T20:46:15.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20382025-11-12T20:46:15.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20392025-11-12T20:46:15.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20402025-11-12T20:46:15.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20412025-11-12T20:46:15.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20422025-11-12T20:46:15.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20432025-11-12T20:46:15.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20442025-11-12T20:46:15.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
20452025-11-12T20:46:15.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20462025-11-12T20:46:15.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
20472025-11-12T20:46:15.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20482025-11-12T20:46:15.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20492025-11-12T20:46:15.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20502025-11-12T20:46:15.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20512025-11-12T20:46:15.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20522025-11-12T20:46:15.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20532025-11-12T20:46:15.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20542025-11-12T20:46:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
20552025-11-12T20:46:15.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20562025-11-12T20:46:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20572025-11-12T20:46:15.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20582025-11-12T20:46:15.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20592025-11-12T20:46:15.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20602025-11-12T20:46:15.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20612025-11-12T20:46:15.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20622025-11-12T20:46:15.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20632025-11-12T20:46:15.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20642025-11-12T20:46:15.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20652025-11-12T20:46:15.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20662025-11-12T20:46:15.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20672025-11-12T20:46:15.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20682025-11-12T20:46:15.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20692025-11-12T20:46:15.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20702025-11-12T20:46:15.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20712025-11-12T20:46:15.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20722025-11-12T20:46:15.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20732025-11-12T20:46:15.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20742025-11-12T20:46:15.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
20752025-11-12T20:46:15.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20762025-11-12T20:46:15.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20772025-11-12T20:46:15.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20782025-11-12T20:46:15.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20792025-11-12T20:46:15.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20802025-11-12T20:46:15.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20812025-11-12T20:46:15.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
20822025-11-12T20:46:15.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20832025-11-12T20:46:15.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20842025-11-12T20:46:15.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
20852025-11-12T20:46:15.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20862025-11-12T20:46:15.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20872025-11-12T20:46:15.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20882025-11-12T20:46:15.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20892025-11-12T20:46:15.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20902025-11-12T20:46:15.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20912025-11-12T20:46:15.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20922025-11-12T20:46:15.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20932025-11-12T20:46:15.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20942025-11-12T20:46:15.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20952025-11-12T20:46:15.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20962025-11-12T20:46:15.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
20972025-11-12T20:46:15.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
20982025-11-12T20:46:15.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
20992025-11-12T20:46:15.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21002025-11-12T20:46:15.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21012025-11-12T20:46:15.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21022025-11-12T20:46:15.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21032025-11-12T20:46:15.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21042025-11-12T20:46:15.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21052025-11-12T20:46:15.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21062025-11-12T20:46:15.597ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
21072025-11-12T20:46:15.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21082025-11-12T20:46:15.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21092025-11-12T20:46:15.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21102025-11-12T20:46:15.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21112025-11-12T20:46:15.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21122025-11-12T20:46:15.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21132025-11-12T20:46:15.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21142025-11-12T20:46:15.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21152025-11-12T20:46:15.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21162025-11-12T20:46:15.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21172025-11-12T20:46:15.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
21182025-11-12T20:46:15.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21192025-11-12T20:46:15.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21202025-11-12T20:46:15.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
21212025-11-12T20:46:15.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21222025-11-12T20:46:15.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21232025-11-12T20:46:15.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21242025-11-12T20:46:15.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21252025-11-12T20:46:15.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21262025-11-12T20:46:15.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21272025-11-12T20:46:15.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21282025-11-12T20:46:15.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21292025-11-12T20:46:15.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21302025-11-12T20:46:15.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21312025-11-12T20:46:15.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21322025-11-12T20:46:15.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21332025-11-12T20:46:15.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21342025-11-12T20:46:15.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21352025-11-12T20:46:15.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21362025-11-12T20:46:15.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21372025-11-12T20:46:15.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21382025-11-12T20:46:15.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
21392025-11-12T20:46:15.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21402025-11-12T20:46:15.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21412025-11-12T20:46:15.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21422025-11-12T20:46:15.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21432025-11-12T20:46:15.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21442025-11-12T20:46:15.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21452025-11-12T20:46:15.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21462025-11-12T20:46:15.646ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
21472025-11-12T20:46:15.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21482025-11-12T20:46:15.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21492025-11-12T20:46:15.657ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
21502025-11-12T20:46:15.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21512025-11-12T20:46:15.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21522025-11-12T20:46:15.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21532025-11-12T20:46:15.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
21542025-11-12T20:46:15.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21552025-11-12T20:46:15.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21562025-11-12T20:46:15.662ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
21572025-11-12T20:46:15.662ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
21582025-11-12T20:46:15.662ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
21592025-11-12T20:46:15.676ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
21602025-11-12T20:46:15.676ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
21612025-11-12T20:46:15.684ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 4ced57e1-d471-496a-85ce-18d7d7498a1c parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
21622025-11-12T20:46:15.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21632025-11-12T20:46:15.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21642025-11-12T20:46:15.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21652025-11-12T20:46:15.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
21662025-11-12T20:46:15.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21672025-11-12T20:46:15.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21682025-11-12T20:46:15.697ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
21692025-11-12T20:46:15.699ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
21702025-11-12T20:46:15.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 381.654787ms iteration = 5 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
21712025-11-12T20:46:15.725ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
21722025-11-12T20:46:15.726ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
21732025-11-12T20:46:15.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21742025-11-12T20:46:15.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21752025-11-12T20:46:15.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21762025-11-12T20:46:15.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21772025-11-12T20:46:15.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21782025-11-12T20:46:15.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
21792025-11-12T20:46:15.765ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21802025-11-12T20:46:15.765ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ac260606-827a-46fa-b3f2-3ea0ee9e4f66", "date": "Wed, 12 Nov 2025 20:46:15 GMT"} })
21812025-11-12T20:46:15.766ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
21822025-11-12T20:46:15.766ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = Dns step id = Ensure
21832025-11-12T20:46:15.766ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
21842025-11-12T20:46:15.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21852025-11-12T20:46:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21862025-11-12T20:46:15.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21872025-11-12T20:46:15.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21882025-11-12T20:46:15.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21892025-11-12T20:46:15.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
21902025-11-12T20:46:15.806ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
21912025-11-12T20:46:15.806ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
21922025-11-12T20:46:15.806ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
21932025-11-12T20:46:15.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21942025-11-12T20:46:15.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21952025-11-12T20:46:15.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21962025-11-12T20:46:15.811ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
21972025-11-12T20:46:15.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
21982025-11-12T20:46:15.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
21992025-11-12T20:46:15.849ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
22002025-11-12T20:46:15.849ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22012025-11-12T20:46:15.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22022025-11-12T20:46:15.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22032025-11-12T20:46:15.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22042025-11-12T20:46:15.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22052025-11-12T20:46:15.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22062025-11-12T20:46:15.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22072025-11-12T20:46:15.858ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
22082025-11-12T20:46:15.858ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
22092025-11-12T20:46:15.858ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
22102025-11-12T20:46:15.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22112025-11-12T20:46:15.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22122025-11-12T20:46:15.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22132025-11-12T20:46:15.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22142025-11-12T20:46:15.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22152025-11-12T20:46:15.863ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22162025-11-12T20:46:15.865ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
22172025-11-12T20:46:15.865ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
22182025-11-12T20:46:15.865ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
22192025-11-12T20:46:15.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22202025-11-12T20:46:15.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22212025-11-12T20:46:15.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22222025-11-12T20:46:15.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22232025-11-12T20:46:15.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22242025-11-12T20:46:15.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22252025-11-12T20:46:15.906ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
22262025-11-12T20:46:15.906ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
22272025-11-12T20:46:15.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22282025-11-12T20:46:15.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22292025-11-12T20:46:15.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22302025-11-12T20:46:15.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22312025-11-12T20:46:15.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22322025-11-12T20:46:15.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22332025-11-12T20:46:15.914ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
22342025-11-12T20:46:15.915ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
22352025-11-12T20:46:15.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22362025-11-12T20:46:15.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22372025-11-12T20:46:15.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22382025-11-12T20:46:15.922ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22392025-11-12T20:46:15.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22402025-11-12T20:46:15.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22412025-11-12T20:46:15.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22422025-11-12T20:46:15.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22432025-11-12T20:46:15.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22442025-11-12T20:46:15.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22452025-11-12T20:46:15.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22462025-11-12T20:46:15.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
22472025-11-12T20:46:15.932ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
22482025-11-12T20:46:15.932ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
22492025-11-12T20:46:15.932ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = OmicronZones step id = Cleanup
22502025-11-12T20:46:15.932ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = OmicronZones step id = Cleanup
22512025-11-12T20:46:15.932ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = PhysicalDisks step id = Cleanup
22522025-11-12T20:46:15.932ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = Clickhouse step id = Ensure
22532025-11-12T20:46:15.932ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = Clickhouse step id = Ensure
22542025-11-12T20:46:15.965ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
22552025-11-12T20:46:15.970ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "81d9e804-6c35-46ff-a1d4-35b9b7cb6ad3", "date": "Wed, 12 Nov 2025 20:46:15 GMT"} })
22562025-11-12T20:46:15.971ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = SupportBundles step id = Cleanup
22572025-11-12T20:46:15.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22582025-11-12T20:46:15.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22592025-11-12T20:46:15.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22602025-11-12T20:46:15.974ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22612025-11-12T20:46:15.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22622025-11-12T20:46:15.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22632025-11-12T20:46:15.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22642025-11-12T20:46:15.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22652025-11-12T20:46:15.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22662025-11-12T20:46:15.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
22672025-11-12T20:46:16.026ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = OmicronZones step id = Cleanup
22682025-11-12T20:46:16.026ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
22692025-11-12T20:46:16.026ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
22702025-11-12T20:46:16.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22712025-11-12T20:46:16.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22722025-11-12T20:46:16.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22732025-11-12T20:46:16.030ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22742025-11-12T20:46:16.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22752025-11-12T20:46:16.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22762025-11-12T20:46:16.063ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
22772025-11-12T20:46:16.065ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
22782025-11-12T20:46:16.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = Cockroach step id = Ensure
22792025-11-12T20:46:16.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22802025-11-12T20:46:16.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22812025-11-12T20:46:16.076ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
22822025-11-12T20:46:16.076ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4e772352-3665-4236-b04e-43d6273110cd step component = MgsUpdates step id = Ensure
22832025-11-12T20:46:16.076ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Dependency
22842025-11-12T20:46:16.076ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 3 reason = Dependency
22852025-11-12T20:46:16.076ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
22862025-11-12T20:46:16.076ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22872025-11-12T20:46:16.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
22882025-11-12T20:46:16.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22892025-11-12T20:46:16.077ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22902025-11-12T20:46:16.077ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22912025-11-12T20:46:16.078ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 760.539492ms iteration = 2 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2258278,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":63487273,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":63487223,"secs":0}},"event_index":1,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":69306833,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":10393039,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":10392999,"secs":0}},"event_index":2,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":79725843,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51302258,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":51302228,"secs":0}},"event_index":3,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":131038061,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40189910,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":40189820,"secs":0}},"event_index":4,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":171240221,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":271014643,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":271014603,"secs":0}},"event_index":5,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":442268285,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":165637506,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":165637456,"secs":0}},"event_index":6,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":607919771,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4371,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4331,"secs":0}},"event_index":7,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":607934922,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":18860,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":18540,"secs":0}},"event_index":8,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":607955682,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3220,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3180,"secs":0}},"event_index":9,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":607963932,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1611,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1580,"secs":0}},"event_index":10,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":607966972,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38560207,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38560167,"secs":0}},"event_index":11,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":646528481,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":55202018,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":55201978,"secs":0}},"event_index":12,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":701740269,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39326407,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":39326367,"secs":0}},"event_index":13,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":741083027,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":10383758,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":10383718,"secs":0}},"event_index":14,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":751476686,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5130,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5100,"secs":0}},"event_index":15,"execution_id":"27717b55-df24-47ac-8aac-4ccd8f247e97","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":751488817,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
22922025-11-12T20:46:16.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
22932025-11-12T20:46:16.080ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
22942025-11-12T20:46:16.080ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
22952025-11-12T20:46:16.080ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22962025-11-12T20:46:16.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
22972025-11-12T20:46:16.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22982025-11-12T20:46:16.083ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(3ea6c7e4-601e-4623-bf5d-213cbc0f74e1, 0), (d993b9ec-5ee5-4ecb-89c3-78c5f40add8e, 0), (af59bb51-5443-4af7-b670-a3bb2595f0bc, 0)]
22992025-11-12T20:46:16.083ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 971, tv_nsec: 580835528 } })
23002025-11-12T20:46:16.114ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23012025-11-12T20:46:16.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23022025-11-12T20:46:16.114ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 971, tv_nsec: 611855871 } })
23032025-11-12T20:46:16.115ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
23042025-11-12T20:46:16.115ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
23052025-11-12T20:46:16.115ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
23062025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e9bd25b-ca65-44e3-a0da-9fefd60febd4", "content-length": "476", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23072025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
23082025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23092025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbe26562-1e25-4da2-a592-151c0c11a14a", "content-length": "734", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23102025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
23112025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbf0699c-7ef4-409a-99ea-d2157be23fdb", "content-length": "111", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23122025-11-12T20:46:16.116ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
23132025-11-12T20:46:16.117ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a433ce1a-69d0-4215-8b5c-368fffba531a", "content-length": "111", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23142025-11-12T20:46:16.117ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
23152025-11-12T20:46:16.117ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f625536d-1dab-4ad5-bd85-dc41d38b1d8a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23162025-11-12T20:46:16.117ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
23172025-11-12T20:46:16.117ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "529960d5-a677-4951-985c-aa6fc37a78d5", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23182025-11-12T20:46:16.117ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
23192025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78a6129c-82df-41d3-919c-5444cedb4e66", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23202025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
23212025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23222025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23232025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2da0e4f1-2838-4049-b54e-a76eb86439b5", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23242025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
23252025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23262025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ed40e06-61d5-454c-87b4-f5b531f23d7c", "content-length": "702", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23272025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
23282025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aedfbda9-acb7-46e4-a18b-05e0119c7009", "content-length": "727", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23292025-11-12T20:46:16.118ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
23302025-11-12T20:46:16.119ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "baeca1da-d83f-4ef4-92ab-7cb53c309647", "content-length": "729", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23312025-11-12T20:46:16.119ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
23322025-11-12T20:46:16.119ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09ba42e2-51c7-43ef-987c-c87c661654e2", "content-length": "728", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23332025-11-12T20:46:16.119ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
23342025-11-12T20:46:16.119ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14287005-fc19-4c65-9c30-087d2984ef79", "content-length": "722", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23352025-11-12T20:46:16.119ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
23362025-11-12T20:46:16.120ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "06e06bb3-de98-40e1-bd95-ccc4ef5334b9", "content-length": "10", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23372025-11-12T20:46:16.120ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
23382025-11-12T20:46:16.120ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ea4f16b-a6c7-4647-8b48-55ebc889119b", "content-length": "144", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23392025-11-12T20:46:16.120ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
23402025-11-12T20:46:16.120ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70e29124-954f-419b-bba6-fd1ab973e3ee", "content-length": "147", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23412025-11-12T20:46:16.120ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
23422025-11-12T20:46:16.121ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a19c8a9f-1456-4236-9fa9-2876f53535ce", "content-length": "109", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23432025-11-12T20:46:16.121ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
23442025-11-12T20:46:16.121ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
23452025-11-12T20:46:16.121ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 45.327253ms iteration = 3 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
23462025-11-12T20:46:16.122ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad2e7fb8-4a7b-4d6a-8aec-6d9d266a2988", "content-length": "109", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23472025-11-12T20:46:16.122ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
23482025-11-12T20:46:16.122ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed6f213f-ed8b-432c-a1af-76ea52000671", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23492025-11-12T20:46:16.122ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
23502025-11-12T20:46:16.123ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1463023d-6bcb-469e-9da4-cb2196494b7c", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23512025-11-12T20:46:16.123ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
23522025-11-12T20:46:16.123ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a246d5f-cafe-43f1-8348-6893bf88d36c", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23532025-11-12T20:46:16.123ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
23542025-11-12T20:46:16.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bbfd985-519e-4d51-bf55-181592dd50fb", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23552025-11-12T20:46:16.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
23562025-11-12T20:46:16.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08e52a62-2ef7-4dd1-89a1-ecbfd5cd8e35", "content-length": "702", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23572025-11-12T20:46:16.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
23582025-11-12T20:46:16.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a4bec1-4540-4989-8c8c-61d89b74547b", "content-length": "727", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23592025-11-12T20:46:16.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
23602025-11-12T20:46:16.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d273485e-580a-4859-855c-e7b4ce9750e2", "content-length": "729", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23612025-11-12T20:46:16.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
23622025-11-12T20:46:16.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cc07c9a-8de5-42be-9fc0-cb09f4261fe5", "content-length": "728", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23632025-11-12T20:46:16.125ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
23642025-11-12T20:46:16.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
23652025-11-12T20:46:16.127ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23662025-11-12T20:46:16.130ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
23672025-11-12T20:46:16.130ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
23682025-11-12T20:46:16.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
23692025-11-12T20:46:16.132ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23702025-11-12T20:46:16.135ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
23712025-11-12T20:46:16.135ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
23722025-11-12T20:46:16.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
23732025-11-12T20:46:16.168ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
23742025-11-12T20:46:16.168ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
23752025-11-12T20:46:16.263ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d82d7f9-4dfc-4f42-89e0-1ebcbadca069", "content-length": "95174", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23762025-11-12T20:46:16.284ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
23772025-11-12T20:46:16.284ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
23782025-11-12T20:46:16.284ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
23792025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
23802025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
23812025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
23822025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
23832025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
23842025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "904754cb-03ec-4e50-a4ed-091cb28616cd", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23852025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
23862025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4afae97a-ecd4-4321-968c-ea115983c664", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23872025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b1cb23d-f201-44e4-8fb8-4cc088f01c4a", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23882025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7950eda-b161-40b9-9827-f96d4d9b8d5e", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23892025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "10dc9bf1-fe6c-4e62-94e0-c410b013c1c3", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23902025-11-12T20:46:16.284ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
23912025-11-12T20:46:16.285ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
23922025-11-12T20:46:16.285ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
23932025-11-12T20:46:16.285ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
23942025-11-12T20:46:16.324ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c87de9b4-cad8-4787-93dd-10dd5c4d7e4a", "content-length": "407000", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23952025-11-12T20:46:16.325ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f796eb9d-c744-4033-b593-a5567a13def7", "content-length": "367367", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23962025-11-12T20:46:16.327ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "12a65099-3c3c-4cb0-9fca-3fd607d3faa7", "content-length": "367724", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23972025-11-12T20:46:16.329ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "730442f1-0495-4bc3-89bf-d37d3b6b790c", "content-length": "365221", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23982025-11-12T20:46:16.329ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "146dc462-70f4-4175-a51f-b9f287ae8178", "content-length": "371993", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
23992025-11-12T20:46:16.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
24002025-11-12T20:46:16.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
24012025-11-12T20:46:16.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
24022025-11-12T20:46:16.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
24032025-11-12T20:46:16.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
24042025-11-12T20:46:16.363ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
24052025-11-12T20:46:16.363ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
24062025-11-12T20:46:16.367ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "d065799c-6995-4b3a-b94f-32432b4160fa", "content-length": "125", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
24072025-11-12T20:46:16.369ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
24082025-11-12T20:46:16.459ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
24092025-11-12T20:46:16.459ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
24102025-11-12T20:46:16.459ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae15136c-2167-4e56-ac99-942155e92cb7", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
24112025-11-12T20:46:16.461ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
24122025-11-12T20:46:16.463ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
24132025-11-12T20:46:16.463ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
24142025-11-12T20:46:16.464ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9081bcc1-00ca-4aba-aff9-2cffcb8db5a0", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
24152025-11-12T20:46:16.466ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
24162025-11-12T20:46:16.467ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
24172025-11-12T20:46:16.467ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
24182025-11-12T20:46:16.468ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cd4d249-0e30-42f6-94bf-fbc38b17acf6", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:46:16 GMT"} })
24192025-11-12T20:46:16.468ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
24202025-11-12T20:46:16.469ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
24212025-11-12T20:46:16.469ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
24222025-11-12T20:46:16.471ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24232025-11-12T20:46:16.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24242025-11-12T20:46:16.473ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24252025-11-12T20:46:16.475ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
24262025-11-12T20:46:16.477ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24272025-11-12T20:46:16.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
24282025-11-12T20:46:16.711ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1251654b-1450-4a6f-ae07-c188425653da file = nexus/db-queries/src/db/datastore/inventory.rs:1614
24292025-11-12T20:46:16.711ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1251654b-1450-4a6f-ae07-c188425653da time_started = 2025-11-12 20:46:16.115468 UTC
24302025-11-12T20:46:16.711ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 5 reason = Dependency
24312025-11-12T20:46:16.711ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
24322025-11-12T20:46:16.711ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 635.084736ms iteration = 2 status = {"collection_id":"1251654b-1450-4a6f-ae07-c188425653da","time_done":"2025-11-12 20:46:16.469973 UTC","time_started":"2025-11-12 20:46:16.115468 UTC"}
24332025-11-12T20:46:16.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
24342025-11-12T20:46:16.713ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24352025-11-12T20:46:16.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
24362025-11-12T20:46:16.716ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
24372025-11-12T20:46:16.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
24382025-11-12T20:46:16.718ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
24392025-11-12T20:46:16.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
24402025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Dependency
24412025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 57.311423ms iteration = 5 status = {"Loaded":{"collection_id":"1251654b-1450-4a6f-ae07-c188425653da","time_loaded":"2025-11-12T20:46:16.711153038Z","time_started":"2025-11-12T20:46:16.115468Z"}}
24422025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 4 reason = Dependency
24432025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24442025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 6 reason = Dependency
24452025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24462025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24472025-11-12T20:46:16.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24482025-11-12T20:46:16.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24492025-11-12T20:46:16.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24502025-11-12T20:46:16.773ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
24512025-11-12T20:46:16.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24522025-11-12T20:46:16.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24532025-11-12T20:46:16.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24542025-11-12T20:46:16.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24552025-11-12T20:46:16.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24562025-11-12T20:46:16.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24572025-11-12T20:46:16.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24582025-11-12T20:46:16.775ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24592025-11-12T20:46:16.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24602025-11-12T20:46:16.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24612025-11-12T20:46:16.776ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
24622025-11-12T20:46:16.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24632025-11-12T20:46:16.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24642025-11-12T20:46:16.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24652025-11-12T20:46:16.777ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24662025-11-12T20:46:16.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24672025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24682025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
24692025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24702025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24712025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24722025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24732025-11-12T20:46:16.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24742025-11-12T20:46:16.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24752025-11-12T20:46:16.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24762025-11-12T20:46:16.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
24772025-11-12T20:46:16.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24782025-11-12T20:46:16.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24792025-11-12T20:46:16.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
24802025-11-12T20:46:16.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24812025-11-12T20:46:16.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24822025-11-12T20:46:16.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24832025-11-12T20:46:16.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24842025-11-12T20:46:16.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24852025-11-12T20:46:16.786ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
24862025-11-12T20:46:16.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24872025-11-12T20:46:16.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24882025-11-12T20:46:16.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24892025-11-12T20:46:16.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
24902025-11-12T20:46:16.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24912025-11-12T20:46:16.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24922025-11-12T20:46:16.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24932025-11-12T20:46:16.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
24942025-11-12T20:46:16.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24952025-11-12T20:46:16.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24962025-11-12T20:46:16.799ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
24972025-11-12T20:46:16.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
24982025-11-12T20:46:16.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
24992025-11-12T20:46:16.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25002025-11-12T20:46:16.803ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
25012025-11-12T20:46:16.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
25022025-11-12T20:46:16.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
25032025-11-12T20:46:16.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25042025-11-12T20:46:16.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
25052025-11-12T20:46:16.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
25062025-11-12T20:46:16.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
25072025-11-12T20:46:16.809ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
25082025-11-12T20:46:16.809ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 41.093444ms iteration = 4 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"1251654b-1450-4a6f-ae07-c188425653da","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
25092025-11-12T20:46:16.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25102025-11-12T20:46:16.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25112025-11-12T20:46:16.831ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 63.165935ms iteration = 5 status = {"physical_disks_added":0}
25122025-11-12T20:46:16.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25132025-11-12T20:46:16.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25142025-11-12T20:46:16.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25152025-11-12T20:46:16.838ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
25162025-11-12T20:46:16.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25172025-11-12T20:46:16.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25182025-11-12T20:46:16.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25192025-11-12T20:46:16.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25202025-11-12T20:46:16.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25212025-11-12T20:46:16.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
25222025-11-12T20:46:16.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25232025-11-12T20:46:16.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25242025-11-12T20:46:16.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25252025-11-12T20:46:16.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
25262025-11-12T20:46:16.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25272025-11-12T20:46:16.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25282025-11-12T20:46:16.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25292025-11-12T20:46:16.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25302025-11-12T20:46:16.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25312025-11-12T20:46:16.856ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
25322025-11-12T20:46:16.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25332025-11-12T20:46:16.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25342025-11-12T20:46:16.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25352025-11-12T20:46:16.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25362025-11-12T20:46:16.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25372025-11-12T20:46:16.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
25382025-11-12T20:46:16.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25392025-11-12T20:46:16.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25402025-11-12T20:46:16.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25412025-11-12T20:46:16.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25422025-11-12T20:46:16.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25432025-11-12T20:46:16.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
25442025-11-12T20:46:16.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25452025-11-12T20:46:16.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25462025-11-12T20:46:16.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25472025-11-12T20:46:16.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25482025-11-12T20:46:16.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25492025-11-12T20:46:16.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25502025-11-12T20:46:16.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25512025-11-12T20:46:16.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
25522025-11-12T20:46:16.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25532025-11-12T20:46:16.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25542025-11-12T20:46:16.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25552025-11-12T20:46:16.904ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
25562025-11-12T20:46:16.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25572025-11-12T20:46:16.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25582025-11-12T20:46:16.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25592025-11-12T20:46:16.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25602025-11-12T20:46:16.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25612025-11-12T20:46:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25622025-11-12T20:46:16.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25632025-11-12T20:46:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
25642025-11-12T20:46:16.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25652025-11-12T20:46:16.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25662025-11-12T20:46:16.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25672025-11-12T20:46:16.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
25682025-11-12T20:46:16.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25692025-11-12T20:46:16.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25702025-11-12T20:46:16.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25712025-11-12T20:46:16.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25722025-11-12T20:46:16.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25732025-11-12T20:46:16.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25742025-11-12T20:46:16.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25752025-11-12T20:46:16.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
25762025-11-12T20:46:16.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25772025-11-12T20:46:16.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25782025-11-12T20:46:16.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25792025-11-12T20:46:16.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25802025-11-12T20:46:16.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25812025-11-12T20:46:16.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
25822025-11-12T20:46:16.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25832025-11-12T20:46:16.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25842025-11-12T20:46:16.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25852025-11-12T20:46:16.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25862025-11-12T20:46:16.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25872025-11-12T20:46:16.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
25882025-11-12T20:46:16.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25892025-11-12T20:46:16.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25902025-11-12T20:46:16.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25912025-11-12T20:46:16.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25922025-11-12T20:46:16.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25932025-11-12T20:46:16.949ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
25942025-11-12T20:46:16.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25952025-11-12T20:46:16.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25962025-11-12T20:46:16.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25972025-11-12T20:46:16.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25982025-11-12T20:46:16.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25992025-11-12T20:46:16.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
26002025-11-12T20:46:16.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26012025-11-12T20:46:16.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26022025-11-12T20:46:16.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26032025-11-12T20:46:16.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26042025-11-12T20:46:16.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26052025-11-12T20:46:16.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
26062025-11-12T20:46:16.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26072025-11-12T20:46:16.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26082025-11-12T20:46:17.016ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
26092025-11-12T20:46:17.016ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
26102025-11-12T20:46:17.016ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = c59bfb08-6bdc-4e19-a168-28629450a698 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
26112025-11-12T20:46:17.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26122025-11-12T20:46:17.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26132025-11-12T20:46:17.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26142025-11-12T20:46:17.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
26152025-11-12T20:46:17.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26162025-11-12T20:46:17.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26172025-11-12T20:46:17.024ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
26182025-11-12T20:46:17.024ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
26192025-11-12T20:46:17.024ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 256.302603ms iteration = 6 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
26202025-11-12T20:46:20.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 6 reason = Timeout
26212025-11-12T20:46:20.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
26222025-11-12T20:46:20.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
26232025-11-12T20:46:20.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 10 reason = Timeout
26242025-11-12T20:46:20.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26252025-11-12T20:46:20.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26262025-11-12T20:46:20.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26272025-11-12T20:46:20.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26282025-11-12T20:46:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
26292025-11-12T20:46:20.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26302025-11-12T20:46:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
26312025-11-12T20:46:20.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
26322025-11-12T20:46:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26332025-11-12T20:46:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
26342025-11-12T20:46:20.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 4 reason = Timeout
26352025-11-12T20:46:20.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
26362025-11-12T20:46:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
26372025-11-12T20:46:20.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
26382025-11-12T20:46:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.908795ms iteration = 10 status = {"config_updated":false}
26392025-11-12T20:46:20.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1251654b-1450-4a6f-ae07-c188425653da original_time_started = 2025-11-12 20:46:16.115468 UTC
26402025-11-12T20:46:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.715095ms iteration = 6 status = {"Loaded":{"collection_id":"1251654b-1450-4a6f-ae07-c188425653da","time_loaded":"2025-11-12T20:46:20.276804487Z","time_started":"2025-11-12T20:46:16.115468Z"}}
26412025-11-12T20:46:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26422025-11-12T20:46:20.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
26432025-11-12T20:46:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
26442025-11-12T20:46:20.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
26452025-11-12T20:46:20.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 4 reason = Timeout
26462025-11-12T20:46:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26472025-11-12T20:46:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26482025-11-12T20:46:20.290ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.907482ms iteration = 4 status = "NoSitrep"
26492025-11-12T20:46:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26502025-11-12T20:46:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26512025-11-12T20:46:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26522025-11-12T20:46:20.293ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
26532025-11-12T20:46:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26542025-11-12T20:46:20.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26552025-11-12T20:46:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26562025-11-12T20:46:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26572025-11-12T20:46:20.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26582025-11-12T20:46:20.299ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
26592025-11-12T20:46:20.300ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
26602025-11-12T20:46:20.300ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.42058ms iteration = 4 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
26612025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
26622025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
26632025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26642025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26652025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 11 reason = Timeout
26662025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26672025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26682025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26692025-11-12T20:46:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26702025-11-12T20:46:25.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26712025-11-12T20:46:25.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26722025-11-12T20:46:25.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26732025-11-12T20:46:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26742025-11-12T20:46:25.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.783118ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26752025-11-12T20:46:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26762025-11-12T20:46:25.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26772025-11-12T20:46:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26782025-11-12T20:46:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
26792025-11-12T20:46:25.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26802025-11-12T20:46:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26812025-11-12T20:46:25.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26822025-11-12T20:46:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26832025-11-12T20:46:25.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
26842025-11-12T20:46:25.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.822222ms iteration = 11 status = {"config_updated":false}
26852025-11-12T20:46:25.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26862025-11-12T20:46:25.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
26872025-11-12T20:46:25.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26882025-11-12T20:46:25.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26892025-11-12T20:46:25.302ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
26902025-11-12T20:46:25.305ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.888107ms iteration = 6 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
26912025-11-12T20:46:30.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 12 reason = Timeout
26922025-11-12T20:46:30.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26932025-11-12T20:46:30.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26942025-11-12T20:46:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26952025-11-12T20:46:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
26962025-11-12T20:46:30.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26972025-11-12T20:46:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26982025-11-12T20:46:30.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
26992025-11-12T20:46:30.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.978691ms iteration = 12 status = {"config_updated":false}
27002025-11-12T20:46:35.274ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
27012025-11-12T20:46:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
27022025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
27032025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
27042025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27052025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
27062025-11-12T20:46:35.275ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
27072025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
27082025-11-12T20:46:35.275ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = ExternalNetworking step id = Ensure
27092025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27102025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
27112025-11-12T20:46:35.275ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
27122025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 14.86\u{b5}s iteration = 3 status = {"error":"no config"}
27132025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
27142025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
27152025-11-12T20:46:35.275ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:36:35.274990973 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
27162025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
27172025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
27182025-11-12T20:46:35.275ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
27192025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 6.74\u{b5}s iteration = 3 status = {"error":"no config"}
27202025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
27212025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 3 reason = Timeout
27222025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
27232025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
27242025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
27252025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
27262025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
27272025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27282025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27292025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27302025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27312025-11-12T20:46:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 990, tv_nsec: 772466883 } })
27322025-11-12T20:46:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 990, tv_nsec: 772495703 } })
27332025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27342025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
27352025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
27362025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
27372025-11-12T20:46:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
27382025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 7 reason = Timeout
27392025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
27402025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.208461ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
27412025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
27422025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
27432025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.314724ms iteration = 2 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
27442025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27452025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
27462025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27472025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27482025-11-12T20:46:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27492025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 7 reason = Timeout
27502025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
27512025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27522025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
27532025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
27542025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
27552025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
27562025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
27572025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
27582025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27592025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
27602025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27612025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
27622025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
27632025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
27642025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
27652025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27662025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27672025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
27682025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27692025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
27702025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
27712025-11-12T20:46:35.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
27722025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27732025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27742025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
27752025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27762025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27772025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
27782025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
27792025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
27802025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27812025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
27822025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 2 reason = Timeout
27832025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27842025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
27852025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
27862025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27872025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 13 reason = Timeout
27882025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27892025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 3 reason = Timeout
27902025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 2 reason = Timeout
27912025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
27922025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
27932025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
27942025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
27952025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
27962025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27972025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
27982025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27992025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
28002025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28012025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
28022025-11-12T20:46:35.277ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
28032025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
28042025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
28052025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28062025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28072025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
28082025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
28092025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28102025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28112025-11-12T20:46:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
28122025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
28132025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
28142025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28152025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28162025-11-12T20:46:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 990, tv_nsec: 774989148 } })
28172025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
28182025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
28192025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
28202025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28212025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
28222025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
28232025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
28242025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
28252025-11-12T20:46:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28262025-11-12T20:46:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28272025-11-12T20:46:35.279ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:36:35.274990973 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
28282025-11-12T20:46:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.077265ms iteration = 2 status = {"expiration":"2025-11-12T20:36:35.274990973Z","pruned":[]}
28292025-11-12T20:46:35.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28302025-11-12T20:46:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28312025-11-12T20:46:35.280ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28322025-11-12T20:46:35.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 990, tv_nsec: 777027451 } })
28332025-11-12T20:46:35.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28342025-11-12T20:46:35.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28352025-11-12T20:46:35.281ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 990, tv_nsec: 778004036 } })
28362025-11-12T20:46:35.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
28372025-11-12T20:46:35.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28382025-11-12T20:46:35.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28392025-11-12T20:46:35.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28402025-11-12T20:46:35.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28412025-11-12T20:46:35.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28422025-11-12T20:46:35.284ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28432025-11-12T20:46:35.284ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.044115ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
28442025-11-12T20:46:35.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 5 reason = Timeout
28452025-11-12T20:46:35.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
28462025-11-12T20:46:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
28472025-11-12T20:46:35.285ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28482025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
28492025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28502025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28512025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28522025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28532025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
28542025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28552025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
28562025-11-12T20:46:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28572025-11-12T20:46:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
28582025-11-12T20:46:35.287ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
28592025-11-12T20:46:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
28602025-11-12T20:46:35.289ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
28612025-11-12T20:46:35.289ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 11.754374ms iteration = 2 status = {"by_rx":{},"error":null}
28622025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28632025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28642025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28652025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 2 reason = Timeout
28662025-11-12T20:46:35.290ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
28672025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 5 reason = Timeout
28682025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
28692025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
28702025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28712025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
28722025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
28732025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
28742025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28752025-11-12T20:46:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
28762025-11-12T20:46:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
28772025-11-12T20:46:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28782025-11-12T20:46:35.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28792025-11-12T20:46:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28802025-11-12T20:46:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
28812025-11-12T20:46:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28822025-11-12T20:46:35.292ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28832025-11-12T20:46:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28842025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
28852025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
28862025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
28872025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28882025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28892025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
28902025-11-12T20:46:35.293ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
28912025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
28922025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
28932025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
28942025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28952025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
28962025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28972025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28982025-11-12T20:46:35.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28992025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
29002025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29012025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29022025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.374507ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
29032025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29042025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29052025-11-12T20:46:35.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29062025-11-12T20:46:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29072025-11-12T20:46:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29082025-11-12T20:46:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29092025-11-12T20:46:35.298ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29102025-11-12T20:46:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29112025-11-12T20:46:35.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
29122025-11-12T20:46:35.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
29132025-11-12T20:46:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
29142025-11-12T20:46:35.300ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1251654b-1450-4a6f-ae07-c188425653da original_time_started = 2025-11-12 20:46:16.115468 UTC
29152025-11-12T20:46:35.300ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 24.861093ms iteration = 7 status = {"Loaded":{"collection_id":"1251654b-1450-4a6f-ae07-c188425653da","time_loaded":"2025-11-12T20:46:35.276089731Z","time_started":"2025-11-12T20:46:16.115468Z"}}
29162025-11-12T20:46:35.301ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29172025-11-12T20:46:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
29182025-11-12T20:46:35.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
29192025-11-12T20:46:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
29202025-11-12T20:46:35.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29212025-11-12T20:46:35.303ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29222025-11-12T20:46:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29232025-11-12T20:46:35.303ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29242025-11-12T20:46:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29252025-11-12T20:46:35.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29262025-11-12T20:46:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
29272025-11-12T20:46:35.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29282025-11-12T20:46:35.307ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29292025-11-12T20:46:35.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
29302025-11-12T20:46:35.307ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29312025-11-12T20:46:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29322025-11-12T20:46:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29332025-11-12T20:46:35.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29342025-11-12T20:46:35.309ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
29352025-11-12T20:46:35.309ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 32.392938ms iteration = 13 status = {"config_updated":false}
29362025-11-12T20:46:35.310ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 990, tv_nsec: 807833308 } })
29372025-11-12T20:46:35.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29382025-11-12T20:46:35.311ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
29392025-11-12T20:46:35.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
29402025-11-12T20:46:35.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29412025-11-12T20:46:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29422025-11-12T20:46:35.313ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.287549ms iteration = 3 status = 0
29432025-11-12T20:46:35.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29442025-11-12T20:46:35.316ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29452025-11-12T20:46:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29462025-11-12T20:46:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29472025-11-12T20:46:35.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29482025-11-12T20:46:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29492025-11-12T20:46:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29502025-11-12T20:46:35.319ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 35.686224ms iteration = 5 status = "NoSitrep"
29512025-11-12T20:46:35.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29522025-11-12T20:46:35.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29532025-11-12T20:46:35.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29542025-11-12T20:46:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
29552025-11-12T20:46:35.326ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
29562025-11-12T20:46:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
29572025-11-12T20:46:35.326ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29582025-11-12T20:46:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
29592025-11-12T20:46:35.328ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
29602025-11-12T20:46:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
29612025-11-12T20:46:35.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29622025-11-12T20:46:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
29632025-11-12T20:46:35.336ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
29642025-11-12T20:46:35.339ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
29652025-11-12T20:46:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
29662025-11-12T20:46:35.344ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
29672025-11-12T20:46:35.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
29682025-11-12T20:46:35.345ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29692025-11-12T20:46:35.348ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29702025-11-12T20:46:35.349ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
29712025-11-12T20:46:35.349ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29722025-11-12T20:46:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29732025-11-12T20:46:35.352ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29742025-11-12T20:46:35.352ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29752025-11-12T20:46:35.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29762025-11-12T20:46:35.352ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29772025-11-12T20:46:35.353ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
29782025-11-12T20:46:35.353ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
29792025-11-12T20:46:35.353ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29802025-11-12T20:46:35.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
29812025-11-12T20:46:35.353ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
29822025-11-12T20:46:35.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29832025-11-12T20:46:35.356ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
29842025-11-12T20:46:35.360ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
29852025-11-12T20:46:35.360ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
29862025-11-12T20:46:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
29872025-11-12T20:46:35.364ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
29882025-11-12T20:46:35.366ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
29892025-11-12T20:46:35.369ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
29902025-11-12T20:46:35.376ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
29912025-11-12T20:46:35.377ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29922025-11-12T20:46:35.378ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
29932025-11-12T20:46:35.379ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29942025-11-12T20:46:35.379ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
29952025-11-12T20:46:35.379ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
29962025-11-12T20:46:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
29972025-11-12T20:46:35.381ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 103.870958ms iteration = 2 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
29982025-11-12T20:46:35.381ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
29992025-11-12T20:46:35.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30002025-11-12T20:46:35.381ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30012025-11-12T20:46:35.381ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
30022025-11-12T20:46:35.382ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 104.593017ms iteration = 7 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
30032025-11-12T20:46:35.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30042025-11-12T20:46:35.382ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = SledList step id = Fetch
30052025-11-12T20:46:35.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30062025-11-12T20:46:35.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30072025-11-12T20:46:35.383ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
30082025-11-12T20:46:35.383ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
30092025-11-12T20:46:35.383ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 105.90311ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
30102025-11-12T20:46:35.384ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30112025-11-12T20:46:35.385ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30122025-11-12T20:46:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30132025-11-12T20:46:35.386ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30142025-11-12T20:46:35.388ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30152025-11-12T20:46:35.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30162025-11-12T20:46:35.388ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30172025-11-12T20:46:35.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
30182025-11-12T20:46:35.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30192025-11-12T20:46:35.391ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
30202025-11-12T20:46:35.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 114.243997ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
30212025-11-12T20:46:35.392ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30222025-11-12T20:46:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30232025-11-12T20:46:35.392ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30242025-11-12T20:46:35.392ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
30252025-11-12T20:46:35.392ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 102.384109ms iteration = 5 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
30262025-11-12T20:46:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30272025-11-12T20:46:35.394ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 117.149902ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
30282025-11-12T20:46:35.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
30292025-11-12T20:46:35.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30302025-11-12T20:46:35.394ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30312025-11-12T20:46:35.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30322025-11-12T20:46:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
30332025-11-12T20:46:35.396ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30342025-11-12T20:46:35.396ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30352025-11-12T20:46:35.396ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 118.964429ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
30362025-11-12T20:46:35.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30372025-11-12T20:46:35.398ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30382025-11-12T20:46:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30392025-11-12T20:46:35.399ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
30402025-11-12T20:46:35.399ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
30412025-11-12T20:46:35.399ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
30422025-11-12T20:46:35.399ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 124.596404ms iteration = 2 status = {"generation":1}
30432025-11-12T20:46:35.399ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
30442025-11-12T20:46:35.404ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30452025-11-12T20:46:35.404ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30462025-11-12T20:46:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30472025-11-12T20:46:35.406ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 129.685316ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30482025-11-12T20:46:35.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30492025-11-12T20:46:35.409ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30502025-11-12T20:46:35.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
30512025-11-12T20:46:35.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 132.985511ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
30522025-11-12T20:46:35.417ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 140.488356ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
30532025-11-12T20:46:35.419ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
30542025-11-12T20:46:35.419ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
30552025-11-12T20:46:35.419ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30562025-11-12T20:46:35.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30572025-11-12T20:46:35.421ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30582025-11-12T20:46:35.423ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
30592025-11-12T20:46:35.423ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 146.551623ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
30602025-11-12T20:46:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30612025-11-12T20:46:35.426ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30622025-11-12T20:46:35.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30632025-11-12T20:46:35.426ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30642025-11-12T20:46:35.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
30652025-11-12T20:46:35.428ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 151.610833ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30662025-11-12T20:46:35.438ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
30672025-11-12T20:46:35.438ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
30682025-11-12T20:46:35.438ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 160.993106ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
30692025-11-12T20:46:35.449ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
30702025-11-12T20:46:35.450ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "b26415cb-4aa9-4d54-89a6-1566bca9ad3d", "content-length": "78", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
30712025-11-12T20:46:35.450ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
30722025-11-12T20:46:35.450ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
30732025-11-12T20:46:35.450ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
30742025-11-12T20:46:35.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
30752025-11-12T20:46:35.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
30762025-11-12T20:46:35.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
30772025-11-12T20:46:35.453ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
30782025-11-12T20:46:35.453ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
30792025-11-12T20:46:35.453ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
30802025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
30812025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
30822025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
30832025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
30842025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
30852025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
30862025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
30872025-11-12T20:46:35.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
30882025-11-12T20:46:35.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
30892025-11-12T20:46:35.459ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
30902025-11-12T20:46:35.459ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
30912025-11-12T20:46:35.459ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
30922025-11-12T20:46:35.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 184.586246ms iteration = 2 status = {"generation":2}
30932025-11-12T20:46:35.459ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
30942025-11-12T20:46:35.467ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
30952025-11-12T20:46:35.467ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 190.038968ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
30962025-11-12T20:46:35.470ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
30972025-11-12T20:46:35.471ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 196.590338ms iteration = 3 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
30982025-11-12T20:46:35.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30992025-11-12T20:46:35.476ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
31002025-11-12T20:46:35.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31012025-11-12T20:46:35.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31022025-11-12T20:46:35.482ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31032025-11-12T20:46:35.482ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
31042025-11-12T20:46:35.482ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31052025-11-12T20:46:35.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
31062025-11-12T20:46:35.486ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
31072025-11-12T20:46:35.486ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31082025-11-12T20:46:35.486ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31092025-11-12T20:46:35.486ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
31102025-11-12T20:46:35.486ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 209.43261ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
31112025-11-12T20:46:35.487ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = DeployNexusRecords step id = Ensure
31122025-11-12T20:46:35.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31132025-11-12T20:46:35.487ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "948188ff-310f-47b7-8053-9128ceddd8e3", "content-length": "26", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
31142025-11-12T20:46:35.487ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31152025-11-12T20:46:35.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31162025-11-12T20:46:35.488ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
31172025-11-12T20:46:35.488ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31182025-11-12T20:46:35.489ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed7f74cf-51f6-4280-8017-7d20db9b7eb3", "content-length": "26", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
31192025-11-12T20:46:35.489ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31202025-11-12T20:46:35.489ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31212025-11-12T20:46:35.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
31222025-11-12T20:46:35.491ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31232025-11-12T20:46:35.492ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31242025-11-12T20:46:35.493ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
31252025-11-12T20:46:35.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31262025-11-12T20:46:35.495ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31272025-11-12T20:46:35.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
31282025-11-12T20:46:35.497ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31292025-11-12T20:46:35.499ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
31302025-11-12T20:46:35.499ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31312025-11-12T20:46:35.500ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f33ae27f-7c1c-4783-aa5a-25b2ad64acfa", "content-length": "372", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
31322025-11-12T20:46:35.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31332025-11-12T20:46:35.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31342025-11-12T20:46:35.502ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31352025-11-12T20:46:35.502ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31362025-11-12T20:46:35.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
31372025-11-12T20:46:35.503ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 226.454591ms iteration = 6 status = {"physical_disks_added":0}
31382025-11-12T20:46:35.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31392025-11-12T20:46:35.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31402025-11-12T20:46:35.512ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
31412025-11-12T20:46:35.512ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 237.300841ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
31422025-11-12T20:46:35.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31432025-11-12T20:46:35.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
31442025-11-12T20:46:35.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31452025-11-12T20:46:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31462025-11-12T20:46:35.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31472025-11-12T20:46:35.526ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
31482025-11-12T20:46:35.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
31492025-11-12T20:46:35.530ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31502025-11-12T20:46:35.531ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
31512025-11-12T20:46:35.531ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
31522025-11-12T20:46:35.531ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
31532025-11-12T20:46:35.531ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
31542025-11-12T20:46:35.531ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "62f88ed3-8c12-4a52-b6e7-d1cae9f245e1", "content-length": "81", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
31552025-11-12T20:46:35.532ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d6a6b39-65bf-4ef4-90fc-30a633270f50", "content-length": "81", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
31562025-11-12T20:46:35.534ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
31572025-11-12T20:46:35.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31582025-11-12T20:46:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31592025-11-12T20:46:35.534ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
31602025-11-12T20:46:35.534ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
31612025-11-12T20:46:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
31622025-11-12T20:46:35.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31632025-11-12T20:46:35.539ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
31642025-11-12T20:46:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
31652025-11-12T20:46:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31662025-11-12T20:46:35.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31672025-11-12T20:46:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31682025-11-12T20:46:35.544ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
31692025-11-12T20:46:35.544ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
31702025-11-12T20:46:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 267.583314ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
31712025-11-12T20:46:35.544ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
31722025-11-12T20:46:35.545ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ed3d0bf-9a89-4a3a-92fa-199f7900d1a4", "content-length": "81", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
31732025-11-12T20:46:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31742025-11-12T20:46:35.546ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 269.506954ms iteration = 3 status = {"disabled":false,"errors":[],"sps":[]}
31752025-11-12T20:46:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
31762025-11-12T20:46:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31772025-11-12T20:46:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31782025-11-12T20:46:35.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31792025-11-12T20:46:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31802025-11-12T20:46:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31812025-11-12T20:46:35.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31822025-11-12T20:46:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
31832025-11-12T20:46:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31842025-11-12T20:46:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31852025-11-12T20:46:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
31862025-11-12T20:46:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31872025-11-12T20:46:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31882025-11-12T20:46:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31892025-11-12T20:46:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31902025-11-12T20:46:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31912025-11-12T20:46:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = / versions = all
31932025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /access versions = all
31942025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /assets/{path} versions = all
31952025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/auth versions = all
31962025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/confirm versions = all
31972025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/success versions = all
31982025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /device/token versions = all
31992025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /device/verify versions = all
32002025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes versions = all
32012025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/probes versions = all
32022025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
32032025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/probes/{probe} versions = all
32042025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles versions = all
32052025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /experimental/v1/system/support-bundles versions = all
32062025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
32072025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
32082025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
32092025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
32102025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
32112025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
32122025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
32132025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
32142025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /images versions = all
32152025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login versions = all
32162025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/local versions = all
32172025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
32182025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
32192025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
32202025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /lookup/{path} versions = all
32212025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects/{path} versions = all
32222025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /projects-new versions = all
32232025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /scim/v2/Groups versions = all
32242025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /scim/v2/Groups versions = all
32252025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /scim/v2/Groups/{group_id} versions = all
32262025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /scim/v2/Groups/{group_id} versions = all
32272025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PATCH path = /scim/v2/Groups/{group_id} versions = all
32282025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /scim/v2/Groups/{group_id} versions = all
32292025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /scim/v2/Users versions = all
32302025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /scim/v2/Users versions = all
32312025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /scim/v2/Users/{user_id} versions = all
32322025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /scim/v2/Users/{user_id} versions = all
32332025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PATCH path = /scim/v2/Users/{user_id} versions = all
32342025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /scim/v2/Users/{user_id} versions = all
32352025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /settings/{path} versions = all
32362025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /system/{path} versions = all
32372025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /utilization versions = all
32382025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups versions = all
32392025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/affinity-groups versions = all
32402025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
32412025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
32422025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
32432025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
32442025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
32452025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
32462025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
32472025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-classes versions = all
32482025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-receivers versions = all
32492025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
32502025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-receivers/{receiver} versions = all
32512025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
32522025-11-12T20:46:35.566ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
32532025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
32542025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
32552025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/alerts/{alert_id}/resend versions = all
32562025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups versions = all
32572025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/anti-affinity-groups versions = all
32582025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
32592025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
32602025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
32612025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
32622025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
32632025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
32642025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
32652025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/auth-settings versions = all
32662025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/auth-settings versions = all
32672025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates versions = all
32682025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/certificates versions = all
32692025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/certificates/{certificate} versions = all
32702025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/certificates/{certificate} versions = all
32712025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks versions = all
32722025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks versions = all
32732025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/disks/{disk} versions = all
32742025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/disks/{disk} versions = all
32752025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
32762025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
32772025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
32782025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/disks/{disk}/finalize versions = all
32792025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips versions = all
32802025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips versions = all
32812025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
32822025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
32832025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
32842025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
32852025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
32862025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups versions = all
32872025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/groups/{group_id} versions = all
32882025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images versions = all
32892025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images versions = all
32902025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/images/{image} versions = all
32912025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/images/{image} versions = all
32922025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/demote versions = all
32932025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/images/{image}/promote versions = all
32942025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances versions = all
32952025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances versions = all
32962025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance} versions = all
32972025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance} versions = all
32982025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/instances/{instance} versions = all
32992025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
33002025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
33012025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/disks versions = all
33022025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
33032025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
33042025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
33052025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
33062025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
33072025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/reboot versions = all
33082025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
33092025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
33102025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
33112025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/start versions = all
33122025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/instances/{instance}/stop versions = all
33132025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
33142025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
33152025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
33162025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
33172025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
33182025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
33192025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways versions = all
33202025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/internet-gateways versions = all
33212025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
33222025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
33232025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools versions = all
33242025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ip-pools/{pool} versions = all
33252025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/login/{silo_name}/local versions = all
33262025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/logout versions = all
33272025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me versions = all
33282025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/access-tokens versions = all
33292025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
33302025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/groups versions = all
33312025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys versions = all
33322025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/me/ssh-keys versions = all
33332025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
33342025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
33352025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/metrics/{metric_name} versions = all
33362025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces versions = all
33372025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/network-interfaces versions = all
33382025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
33392025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/network-interfaces/{interface} versions = all
33402025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
33412025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/ping versions = all
33422025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/policy versions = all
33432025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/policy versions = all
33442025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects versions = all
33452025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/projects versions = all
33462025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/projects/{project} versions = all
33472025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project} versions = all
33482025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project} versions = all
33492025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/projects/{project}/policy versions = all
33502025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/projects/{project}/policy versions = all
33512025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots versions = all
33522025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/snapshots versions = all
33532025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
33542025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/snapshots/{snapshot} versions = all
33552025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/audit-log versions = all
33562025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks versions = all
33572025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
33582025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
33592025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks versions = all
33602025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
33612025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds versions = all
33622025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/sleds versions = all
33632025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
33642025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
33652025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
33662025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
33672025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
33682025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port versions = all
33692025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
33702025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
33712025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
33722025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
33732025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
33742025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches versions = all
33752025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
33762025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers versions = all
33772025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users versions = all
33782025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
33792025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
33802025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/identity-providers/saml versions = all
33812025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
33822025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools versions = all
33832025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools versions = all
33842025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
33852025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
33862025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
33872025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
33882025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
33892025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
33902025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
33912025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
33922025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
33932025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
33942025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
33952025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service versions = all
33962025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
33972025-11-12T20:46:35.567ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
33982025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
33992025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
34002025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot versions = all
34012025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/address-lot versions = all
34022025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
34032025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
34042025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
34052025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/allow-list versions = all
34062025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/allow-list versions = all
34072025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-disable versions = all
34082025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bfd-enable versions = all
34092025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bfd-status versions = all
34102025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp versions = all
34112025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp versions = all
34122025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/bgp versions = all
34132025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
34142025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
34152025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
34162025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
34172025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-exported versions = all
34182025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
34192025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
34202025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/bgp-status versions = all
34212025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/inbound-icmp versions = all
34222025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/networking/inbound-icmp versions = all
34232025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/loopback-address versions = all
34242025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/loopback-address versions = all
34252025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
34262025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
34272025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
34282025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
34292025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
34302025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/policy versions = all
34312025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/policy versions = all
34322025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/scim/tokens versions = all
34332025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/scim/tokens versions = all
34342025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/scim/tokens/{token_id} versions = all
34352025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/scim/tokens/{token_id} versions = all
34362025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silo-quotas versions = all
34372025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos versions = all
34382025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/silos versions = all
34392025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/silos/{silo} versions = all
34402025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo} versions = all
34412025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
34422025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
34432025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
34442025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
34452025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
34462025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/timeseries/query versions = all
34472025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/timeseries/schemas versions = all
34482025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/repositories versions = all
34492025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/repositories versions = all
34502025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/repositories/{system_version} versions = all
34512025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/status versions = all
34522025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/system/update/target-release versions = all
34532025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/trust-roots versions = all
34542025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/system/update/trust-roots versions = all
34552025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
34562025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
34572025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users versions = all
34582025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users/{user_id} versions = all
34592025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin versions = all
34602025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/users-builtin/{user} versions = all
34612025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos versions = all
34622025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
34632025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/timeseries/query versions = all
34642025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users versions = all
34652025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users/{user_id} versions = all
34662025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users/{user_id}/access-tokens versions = all
34672025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/users/{user_id}/logout versions = all
34682025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/users/{user_id}/sessions versions = all
34692025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/utilization versions = all
34702025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-firewall-rules versions = all
34712025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-firewall-rules versions = all
34722025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes versions = all
34732025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-router-routes versions = all
34742025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
34752025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
34762025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
34772025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers versions = all
34782025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-routers versions = all
34792025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
34802025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-routers/{router} versions = all
34812025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-routers/{router} versions = all
34822025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets versions = all
34832025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpc-subnets versions = all
34842025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
34852025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
34862025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
34872025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
34882025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs versions = all
34892025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/vpcs versions = all
34902025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
34912025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/vpcs/{vpc} versions = all
34922025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/vpcs/{vpc} versions = all
34932025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/webhook-receivers versions = all
34942025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
34952025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = GET path = /v1/webhook-secrets versions = all
34962025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = POST path = /v1/webhook-secrets versions = all
34972025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.7:443 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
34982025-11-12T20:46:35.568ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = 172.30.2.7:443
34992025-11-12T20:46:35.568ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.7:443
35002025-11-12T20:46:35.568ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
35012025-11-12T20:46:35.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35022025-11-12T20:46:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35032025-11-12T20:46:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
35042025-11-12T20:46:35.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35052025-11-12T20:46:35.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35062025-11-12T20:46:35.571ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = SledAgent step id = Ensure
35072025-11-12T20:46:35.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35082025-11-12T20:46:35.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
35092025-11-12T20:46:35.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35102025-11-12T20:46:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35112025-11-12T20:46:35.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35122025-11-12T20:46:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35132025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = / versions = all
35142025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /access versions = all
35152025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /assets/{path} versions = all
35162025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/auth versions = all
35172025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/confirm versions = all
35182025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/success versions = all
35192025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /device/token versions = all
35202025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /device/verify versions = all
35212025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes versions = all
35222025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/probes versions = all
35232025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
35242025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
35252025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
35262025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
35272025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
35282025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
35292025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
35302025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
35312025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
35322025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
35332025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
35342025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
35352025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /images versions = all
35362025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login versions = all
35372025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/local versions = all
35382025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
35392025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
35402025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
35412025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /lookup/{path} versions = all
35422025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects/{path} versions = all
35432025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /projects-new versions = all
35442025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /scim/v2/Groups versions = all
35452025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /scim/v2/Groups versions = all
35462025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /scim/v2/Groups/{group_id} versions = all
35472025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /scim/v2/Groups/{group_id} versions = all
35482025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PATCH path = /scim/v2/Groups/{group_id} versions = all
35492025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /scim/v2/Groups/{group_id} versions = all
35502025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /scim/v2/Users versions = all
35512025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /scim/v2/Users versions = all
35522025-11-12T20:46:35.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35532025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /scim/v2/Users/{user_id} versions = all
35542025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /scim/v2/Users/{user_id} versions = all
35552025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PATCH path = /scim/v2/Users/{user_id} versions = all
35562025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /scim/v2/Users/{user_id} versions = all
35572025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /settings/{path} versions = all
35582025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /system/{path} versions = all
35592025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /utilization versions = all
35602025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups versions = all
35612025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/affinity-groups versions = all
35622025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
35632025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
35642025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
35652025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
35662025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
35672025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
35682025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
35692025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-classes versions = all
35702025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-receivers versions = all
35712025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
35722025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-receivers/{receiver} versions = all
35732025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
35742025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
35752025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
35762025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
35772025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/alerts/{alert_id}/resend versions = all
35782025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups versions = all
35792025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/anti-affinity-groups versions = all
35802025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
35812025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
35822025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
35832025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
35842025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
35852025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
35862025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
35872025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/auth-settings versions = all
35882025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/auth-settings versions = all
35892025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates versions = all
35902025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/certificates versions = all
35912025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
35922025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/certificates/{certificate} versions = all
35932025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks versions = all
35942025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks versions = all
35952025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/disks/{disk} versions = all
35962025-11-12T20:46:35.582ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/disks/{disk} versions = all
35972025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
35982025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
35992025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
36002025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
36012025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips versions = all
36022025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips versions = all
36032025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
36042025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
36052025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
36062025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
36072025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
36082025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups versions = all
36092025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/groups/{group_id} versions = all
36102025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images versions = all
36112025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images versions = all
36122025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/images/{image} versions = all
36132025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/images/{image} versions = all
36142025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/demote versions = all
36152025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/images/{image}/promote versions = all
36162025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances versions = all
36172025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances versions = all
36182025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance} versions = all
36192025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance} versions = all
36202025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/instances/{instance} versions = all
36212025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
36222025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
36232025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
36242025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
36252025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
36262025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
36272025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
36282025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
36292025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
36302025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
36312025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
36322025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
36332025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/start versions = all
36342025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
36352025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
36362025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
36372025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
36382025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
36392025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
36402025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
36412025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways versions = all
36422025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/internet-gateways versions = all
36432025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
36442025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
36452025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools versions = all
36462025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
36472025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
36482025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/logout versions = all
36492025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me versions = all
36502025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/access-tokens versions = all
36512025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
36522025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/groups versions = all
36532025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys versions = all
36542025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/me/ssh-keys versions = all
36552025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
36562025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
36572025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
36582025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces versions = all
36592025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/network-interfaces versions = all
36602025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
36612025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
36622025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
36632025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/ping versions = all
36642025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/policy versions = all
36652025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/policy versions = all
36662025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects versions = all
36672025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/projects versions = all
36682025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/projects/{project} versions = all
36692025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project} versions = all
36702025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project} versions = all
36712025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/projects/{project}/policy versions = all
36722025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
36732025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots versions = all
36742025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/snapshots versions = all
36752025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
36762025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
36772025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/audit-log versions = all
36782025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks versions = all
36792025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
36802025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
36812025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks versions = all
36822025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
36832025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds versions = all
36842025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/sleds versions = all
36852025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
36862025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
36872025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
36882025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
36892025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
36902025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
36912025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
36922025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
36932025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
36942025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
36952025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
36962025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches versions = all
36972025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
36982025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers versions = all
36992025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
37002025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
37012025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
37022025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
37032025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
37042025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools versions = all
37052025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools versions = all
37062025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
37072025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
37082025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
37092025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
37102025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
37112025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
37122025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
37132025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
37142025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
37152025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
37162025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
37172025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service versions = all
37182025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
37192025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
37202025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
37212025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
37222025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot versions = all
37232025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/address-lot versions = all
37242025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
37252025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
37262025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
37272025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/allow-list versions = all
37282025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
37292025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
37302025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
37312025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
37322025-11-12T20:46:35.583ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
37332025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp versions = all
37342025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/bgp versions = all
37352025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
37362025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
37372025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
37382025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
37392025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
37402025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
37412025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
37422025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
37432025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/inbound-icmp versions = all
37442025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/networking/inbound-icmp versions = all
37452025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
37462025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
37472025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
37482025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
37492025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
37502025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
37512025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
37522025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/policy versions = all
37532025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/policy versions = all
37542025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/scim/tokens versions = all
37552025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/scim/tokens versions = all
37562025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/scim/tokens/{token_id} versions = all
37572025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/scim/tokens/{token_id} versions = all
37582025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silo-quotas versions = all
37592025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos versions = all
37602025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/silos versions = all
37612025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
37622025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo} versions = all
37632025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
37642025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
37652025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
37662025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
37672025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
37682025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/timeseries/query versions = all
37692025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
37702025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/repositories versions = all
37712025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/repositories versions = all
37722025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/repositories/{system_version} versions = all
37732025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/status versions = all
37742025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/system/update/target-release versions = all
37752025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/trust-roots versions = all
37762025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/system/update/trust-roots versions = all
37772025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
37782025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
37792025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users versions = all
37802025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users/{user_id} versions = all
37812025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin versions = all
37822025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
37832025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos versions = all
37842025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
37852025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/timeseries/query versions = all
37862025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users versions = all
37872025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users/{user_id} versions = all
37882025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users/{user_id}/access-tokens versions = all
37892025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/users/{user_id}/logout versions = all
37902025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/users/{user_id}/sessions versions = all
37912025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/utilization versions = all
37922025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
37932025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
37942025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes versions = all
37952025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-router-routes versions = all
37962025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
37972025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
37982025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
37992025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers versions = all
38002025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-routers versions = all
38012025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
38022025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
38032025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
38042025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets versions = all
38052025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpc-subnets versions = all
38062025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
38072025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
38082025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
38092025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
38102025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs versions = all
38112025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/vpcs versions = all
38122025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
38132025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
38142025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
38152025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/webhook-receivers versions = all
38162025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
38172025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = GET path = /v1/webhook-secrets versions = all
38182025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = POST path = /v1/webhook-secrets versions = all
38192025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::c]:12228 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
38202025-11-12T20:46:35.584ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:12228
38212025-11-12T20:46:35.584ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:12228
38222025-11-12T20:46:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38232025-11-12T20:46:35.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38242025-11-12T20:46:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38252025-11-12T20:46:35.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38262025-11-12T20:46:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38272025-11-12T20:46:35.590ZDEBGnexus (nexus-producer-server): registered DTrace probes
38282025-11-12T20:46:35.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38292025-11-12T20:46:35.594ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::c]:46805 method = GET path = /{producer_id} versions = all
38302025-11-12T20:46:35.595ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::c]:46805
38312025-11-12T20:46:35.595ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::c]:46805
38322025-11-12T20:46:35.595ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::c]:46805
38332025-11-12T20:46:35.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
38342025-11-12T20:46:35.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38352025-11-12T20:46:35.595ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::c]:12221
38362025-11-12T20:46:35.595ZDEBGnexus (nexus-producer-server): starting producer registration task
38372025-11-12T20:46:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38382025-11-12T20:46:35.596ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
38392025-11-12T20:46:35.596ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::c]:46805 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
38402025-11-12T20:46:35.596ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
38412025-11-12T20:46:35.596ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
38422025-11-12T20:46:35.601ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
38432025-11-12T20:46:35.601ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
38442025-11-12T20:46:35.601ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "73d19a6d-6ca1-453c-8b90-a0ed5df163bd", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
38452025-11-12T20:46:35.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38462025-11-12T20:46:35.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38472025-11-12T20:46:35.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38482025-11-12T20:46:35.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38492025-11-12T20:46:35.607ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38502025-11-12T20:46:35.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38512025-11-12T20:46:35.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38522025-11-12T20:46:35.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38532025-11-12T20:46:35.611ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
38542025-11-12T20:46:35.611ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38552025-11-12T20:46:35.611ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "04037e12-1c67-4a61-811e-e97a72a77c92", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
38562025-11-12T20:46:35.613ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 335.939464ms iteration = 4 status = {}
38572025-11-12T20:46:35.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38582025-11-12T20:46:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38592025-11-12T20:46:35.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38602025-11-12T20:46:35.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38612025-11-12T20:46:35.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38622025-11-12T20:46:35.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38632025-11-12T20:46:35.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38642025-11-12T20:46:35.623ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38652025-11-12T20:46:35.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38662025-11-12T20:46:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38672025-11-12T20:46:35.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38682025-11-12T20:46:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38692025-11-12T20:46:35.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38702025-11-12T20:46:35.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38712025-11-12T20:46:35.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38722025-11-12T20:46:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38732025-11-12T20:46:35.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38742025-11-12T20:46:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38752025-11-12T20:46:35.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38762025-11-12T20:46:35.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38772025-11-12T20:46:35.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38782025-11-12T20:46:35.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38792025-11-12T20:46:35.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38802025-11-12T20:46:35.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38812025-11-12T20:46:35.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38822025-11-12T20:46:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38832025-11-12T20:46:35.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38842025-11-12T20:46:35.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
38852025-11-12T20:46:35.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38862025-11-12T20:46:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38872025-11-12T20:46:35.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38882025-11-12T20:46:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38892025-11-12T20:46:35.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38902025-11-12T20:46:35.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
38912025-11-12T20:46:35.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38922025-11-12T20:46:35.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38932025-11-12T20:46:35.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38942025-11-12T20:46:35.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38952025-11-12T20:46:35.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38962025-11-12T20:46:35.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38972025-11-12T20:46:35.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38982025-11-12T20:46:35.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38992025-11-12T20:46:35.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39002025-11-12T20:46:35.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39012025-11-12T20:46:35.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39022025-11-12T20:46:35.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
39032025-11-12T20:46:35.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39042025-11-12T20:46:35.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39052025-11-12T20:46:35.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39062025-11-12T20:46:35.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39072025-11-12T20:46:35.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39082025-11-12T20:46:35.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
39092025-11-12T20:46:35.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39102025-11-12T20:46:35.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39112025-11-12T20:46:35.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39122025-11-12T20:46:35.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39132025-11-12T20:46:35.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39142025-11-12T20:46:35.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
39152025-11-12T20:46:35.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39162025-11-12T20:46:35.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39172025-11-12T20:46:35.691ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
39182025-11-12T20:46:35.693ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
39192025-11-12T20:46:35.693ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
39202025-11-12T20:46:35.693ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = d1002ba6-16fc-44bd-a620-b28fb198e70e parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
39212025-11-12T20:46:35.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39222025-11-12T20:46:35.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39232025-11-12T20:46:35.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39242025-11-12T20:46:35.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
39252025-11-12T20:46:35.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39262025-11-12T20:46:35.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39272025-11-12T20:46:35.701ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
39282025-11-12T20:46:35.701ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
39292025-11-12T20:46:35.701ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 424.583028ms iteration = 7 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
39302025-11-12T20:46:35.714ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
39312025-11-12T20:46:35.714ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "3b312ba3-96bc-41d5-9789-2ab68d667763", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
39322025-11-12T20:46:35.714ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
39332025-11-12T20:46:35.716ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
39342025-11-12T20:46:35.716ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 426.122468ms iteration = 2 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
39352025-11-12T20:46:35.716ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Dependency
39362025-11-12T20:46:35.716ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
39372025-11-12T20:46:35.716ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39382025-11-12T20:46:35.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39392025-11-12T20:46:35.717ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
39402025-11-12T20:46:35.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39412025-11-12T20:46:35.718ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7326742-73cc-4841-8d0a-b681ae09ddd0", "content-length": "691", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
39422025-11-12T20:46:35.718ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
39432025-11-12T20:46:35.718ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
39442025-11-12T20:46:35.718ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
39452025-11-12T20:46:35.720ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39462025-11-12T20:46:35.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39472025-11-12T20:46:35.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39482025-11-12T20:46:35.720ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 443.455227ms iteration = 3 status = {}
39492025-11-12T20:46:35.721ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39502025-11-12T20:46:35.723ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39512025-11-12T20:46:35.723ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39522025-11-12T20:46:35.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39532025-11-12T20:46:35.728ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 225015204 } })
39542025-11-12T20:46:35.742ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 467.298783ms iteration = 3 status = {}
39552025-11-12T20:46:35.781ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
39562025-11-12T20:46:35.782ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "5cd330d3-7b4d-489a-aa97-aea840f27066", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39572025-11-12T20:46:35.782ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39582025-11-12T20:46:35.783ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "19401eee-d578-4038-ad74-258b881dc13e", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39592025-11-12T20:46:35.783ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39602025-11-12T20:46:35.785ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
39612025-11-12T20:46:35.788ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 328.937772ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
39622025-11-12T20:46:35.818ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
39632025-11-12T20:46:35.820ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "d68fb9f9-06dd-4d04-bfb5-e9e26de28f34", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
39642025-11-12T20:46:35.821ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = FirewallRules step id = Ensure
39652025-11-12T20:46:35.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39662025-11-12T20:46:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39672025-11-12T20:46:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39682025-11-12T20:46:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39692025-11-12T20:46:35.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39702025-11-12T20:46:35.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39712025-11-12T20:46:35.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39722025-11-12T20:46:35.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39732025-11-12T20:46:35.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39742025-11-12T20:46:35.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39752025-11-12T20:46:35.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39762025-11-12T20:46:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39772025-11-12T20:46:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39782025-11-12T20:46:35.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
39792025-11-12T20:46:35.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39802025-11-12T20:46:35.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39812025-11-12T20:46:35.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39822025-11-12T20:46:35.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39832025-11-12T20:46:35.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39842025-11-12T20:46:35.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39852025-11-12T20:46:35.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39862025-11-12T20:46:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
39872025-11-12T20:46:35.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39882025-11-12T20:46:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39892025-11-12T20:46:35.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39902025-11-12T20:46:35.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39912025-11-12T20:46:35.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39922025-11-12T20:46:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39932025-11-12T20:46:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39942025-11-12T20:46:35.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39952025-11-12T20:46:35.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39962025-11-12T20:46:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39972025-11-12T20:46:35.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39982025-11-12T20:46:35.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39992025-11-12T20:46:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40002025-11-12T20:46:35.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40012025-11-12T20:46:35.864ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
40022025-11-12T20:46:35.864ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40032025-11-12T20:46:35.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40042025-11-12T20:46:35.865ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "619bccbe-dfe6-4b57-bf10-8c7239826ae5", "content-length": "372", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
40052025-11-12T20:46:35.865ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40062025-11-12T20:46:35.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40072025-11-12T20:46:35.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
40082025-11-12T20:46:35.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40092025-11-12T20:46:35.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40102025-11-12T20:46:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40112025-11-12T20:46:35.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40122025-11-12T20:46:35.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40132025-11-12T20:46:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40142025-11-12T20:46:35.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40152025-11-12T20:46:35.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40162025-11-12T20:46:35.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40172025-11-12T20:46:35.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40182025-11-12T20:46:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40192025-11-12T20:46:35.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40202025-11-12T20:46:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40212025-11-12T20:46:35.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40222025-11-12T20:46:35.885ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
40232025-11-12T20:46:35.885ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:44367
40242025-11-12T20:46:35.886ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44367 req_id = e8033871-4259-46e5-b6c0-f2bde46787b6 uri = /metrics/producers version = 1.0.0
40252025-11-12T20:46:35.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40262025-11-12T20:46:35.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
40272025-11-12T20:46:35.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40282025-11-12T20:46:35.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40292025-11-12T20:46:35.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40302025-11-12T20:46:35.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40312025-11-12T20:46:35.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40322025-11-12T20:46:35.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40332025-11-12T20:46:35.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40342025-11-12T20:46:35.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
40352025-11-12T20:46:35.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40362025-11-12T20:46:35.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40372025-11-12T20:46:35.909ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
40382025-11-12T20:46:35.910ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44367 req_id = e8033871-4259-46e5-b6c0-f2bde46787b6 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
40392025-11-12T20:46:35.910ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44367 req_id = e8033871-4259-46e5-b6c0-f2bde46787b6 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
40402025-11-12T20:46:35.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40412025-11-12T20:46:35.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40422025-11-12T20:46:35.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40432025-11-12T20:46:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40442025-11-12T20:46:35.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40452025-11-12T20:46:35.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40462025-11-12T20:46:35.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40472025-11-12T20:46:35.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40482025-11-12T20:46:35.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40492025-11-12T20:46:35.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40502025-11-12T20:46:35.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40512025-11-12T20:46:35.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40522025-11-12T20:46:35.923ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
40532025-11-12T20:46:35.923ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
40542025-11-12T20:46:35.924ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d881a7d7-2577-4d0c-ba37-e324e815900d", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
40552025-11-12T20:46:35.924ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40562025-11-12T20:46:35.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40572025-11-12T20:46:35.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40582025-11-12T20:46:35.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40592025-11-12T20:46:35.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40602025-11-12T20:46:35.926ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40612025-11-12T20:46:35.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40622025-11-12T20:46:35.929ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40632025-11-12T20:46:35.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40642025-11-12T20:46:35.931ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
40652025-11-12T20:46:35.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40662025-11-12T20:46:35.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40672025-11-12T20:46:35.932ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
40682025-11-12T20:46:35.932ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40692025-11-12T20:46:35.932ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c588d336-fa41-4be2-a813-bf1371ca1027", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
40702025-11-12T20:46:35.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40712025-11-12T20:46:35.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40722025-11-12T20:46:35.934ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 217.870068ms iteration = 5 status = {}
40732025-11-12T20:46:35.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40742025-11-12T20:46:35.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40752025-11-12T20:46:35.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40762025-11-12T20:46:35.943ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
40772025-11-12T20:46:35.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40782025-11-12T20:46:35.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40792025-11-12T20:46:35.950ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
40802025-11-12T20:46:35.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40812025-11-12T20:46:35.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40822025-11-12T20:46:35.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40832025-11-12T20:46:35.953ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40842025-11-12T20:46:35.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40852025-11-12T20:46:35.955ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
40862025-11-12T20:46:35.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40872025-11-12T20:46:35.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40882025-11-12T20:46:35.956ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
40892025-11-12T20:46:35.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40902025-11-12T20:46:35.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40912025-11-12T20:46:35.957ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
40922025-11-12T20:46:35.957ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
40932025-11-12T20:46:35.957ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
40942025-11-12T20:46:35.968ZINFOnexus: assigned collector to new producer collector_id = ecb0f932-e2e8-41d3-85a9-494e5cae5bce file = nexus/src/app/oximeter.rs:76 producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
40952025-11-12T20:46:35.969ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
40962025-11-12T20:46:35.969ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "71ce6db0-f61a-4bac-86b8-4db23cce0841", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
40972025-11-12T20:46:35.970ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
40982025-11-12T20:46:35.972ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "6c47f0bf-1ecc-4680-980e-50fc9a739f5e", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
40992025-11-12T20:46:35.972ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
41002025-11-12T20:46:35.973ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 87668 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44367 req_id = e8033871-4259-46e5-b6c0-f2bde46787b6 response_code = 201 uri = /metrics/producers
41012025-11-12T20:46:35.973ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "e8033871-4259-46e5-b6c0-f2bde46787b6", "content-length": "41", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} })
41022025-11-12T20:46:35.973ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1b53147c-f9af-42de-85b2-744dc6be3d8d", "date": "Wed, 12 Nov 2025 20:46:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
41032025-11-12T20:46:35.973ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
41042025-11-12T20:46:35.975ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
41052025-11-12T20:46:35.975ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
41062025-11-12T20:46:35.975ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 576.289533ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
41072025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
41082025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
41092025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
41102025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
41112025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41122025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41132025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41142025-11-12T20:46:35.985ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41152025-11-12T20:46:35.986ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41162025-11-12T20:46:35.986ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41172025-11-12T20:46:35.986ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
41182025-11-12T20:46:35.986ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41192025-11-12T20:46:35.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41202025-11-12T20:46:35.998ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
41212025-11-12T20:46:35.998ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41222025-11-12T20:46:36.110ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
41232025-11-12T20:46:36.110ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
41242025-11-12T20:46:36.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41252025-11-12T20:46:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41262025-11-12T20:46:36.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41272025-11-12T20:46:36.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41282025-11-12T20:46:36.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41292025-11-12T20:46:36.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
41302025-11-12T20:46:36.151ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
41312025-11-12T20:46:36.152ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "8fcf1a67-e92c-437c-be1f-5b0c249903ad", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
41322025-11-12T20:46:36.153ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
41332025-11-12T20:46:36.153ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = Dns step id = Ensure
41342025-11-12T20:46:36.153ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
41352025-11-12T20:46:36.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41362025-11-12T20:46:36.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41372025-11-12T20:46:36.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41382025-11-12T20:46:36.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41392025-11-12T20:46:36.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41402025-11-12T20:46:36.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41412025-11-12T20:46:36.160ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
41422025-11-12T20:46:36.160ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
41432025-11-12T20:46:36.160ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
41442025-11-12T20:46:36.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41452025-11-12T20:46:36.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41462025-11-12T20:46:36.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41472025-11-12T20:46:36.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41482025-11-12T20:46:36.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41492025-11-12T20:46:36.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41502025-11-12T20:46:36.169ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
41512025-11-12T20:46:36.169ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
41522025-11-12T20:46:36.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41532025-11-12T20:46:36.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41542025-11-12T20:46:36.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41552025-11-12T20:46:36.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41562025-11-12T20:46:36.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41572025-11-12T20:46:36.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41582025-11-12T20:46:36.178ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
41592025-11-12T20:46:36.178ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
41602025-11-12T20:46:36.178ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
41612025-11-12T20:46:36.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41622025-11-12T20:46:36.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41632025-11-12T20:46:36.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41642025-11-12T20:46:36.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41652025-11-12T20:46:36.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41662025-11-12T20:46:36.186ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41672025-11-12T20:46:36.227ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
41682025-11-12T20:46:36.227ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
41692025-11-12T20:46:36.227ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
41702025-11-12T20:46:36.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41712025-11-12T20:46:36.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41722025-11-12T20:46:36.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41732025-11-12T20:46:36.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41742025-11-12T20:46:36.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41752025-11-12T20:46:36.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41762025-11-12T20:46:36.257ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e104b71d-536c-48d8-8fcf-79380ba72318", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
41772025-11-12T20:46:36.257ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
41782025-11-12T20:46:36.257ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
41792025-11-12T20:46:36.264ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0392c70-0272-4e6d-9ab6-10aebbb81892", "content-length": "682", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
41802025-11-12T20:46:36.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41812025-11-12T20:46:36.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41822025-11-12T20:46:36.273ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
41832025-11-12T20:46:36.273ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
41842025-11-12T20:46:36.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41852025-11-12T20:46:36.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41862025-11-12T20:46:36.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41872025-11-12T20:46:36.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
41882025-11-12T20:46:36.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41892025-11-12T20:46:36.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41902025-11-12T20:46:36.281ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
41912025-11-12T20:46:36.281ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
41922025-11-12T20:46:36.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41932025-11-12T20:46:36.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41942025-11-12T20:46:36.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41952025-11-12T20:46:36.285ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
41962025-11-12T20:46:36.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41972025-11-12T20:46:36.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41982025-11-12T20:46:36.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41992025-11-12T20:46:36.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42002025-11-12T20:46:36.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42012025-11-12T20:46:36.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42022025-11-12T20:46:36.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42032025-11-12T20:46:36.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42042025-11-12T20:46:36.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42052025-11-12T20:46:36.292ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42062025-11-12T20:46:36.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42072025-11-12T20:46:36.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
42082025-11-12T20:46:36.295ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
42092025-11-12T20:46:36.295ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
42102025-11-12T20:46:36.295ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = OmicronZones step id = Cleanup
42112025-11-12T20:46:36.295ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = OmicronZones step id = Cleanup
42122025-11-12T20:46:36.295ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = PhysicalDisks step id = Cleanup
42132025-11-12T20:46:36.295ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = Clickhouse step id = Ensure
42142025-11-12T20:46:36.295ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = Clickhouse step id = Ensure
42152025-11-12T20:46:36.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42162025-11-12T20:46:36.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42172025-11-12T20:46:36.309ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.0324871s iteration = 3 status = {}
42182025-11-12T20:46:36.325ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
42192025-11-12T20:46:36.326ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 503, headers: {"content-type": "application/json", "x-request-id": "3daa0d98-8da6-441f-b165-5d8364784808", "content-length": "94", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42202025-11-12T20:46:36.327ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = SupportBundles step id = Cleanup
42212025-11-12T20:46:36.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42222025-11-12T20:46:36.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42232025-11-12T20:46:36.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42242025-11-12T20:46:36.330ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
42252025-11-12T20:46:36.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42262025-11-12T20:46:36.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42272025-11-12T20:46:36.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42282025-11-12T20:46:36.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42292025-11-12T20:46:36.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42302025-11-12T20:46:36.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
42312025-11-12T20:46:36.387ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = OmicronZones step id = Cleanup
42322025-11-12T20:46:36.387ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
42332025-11-12T20:46:36.387ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
42342025-11-12T20:46:36.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42352025-11-12T20:46:36.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42362025-11-12T20:46:36.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42372025-11-12T20:46:36.391ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
42382025-11-12T20:46:36.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42392025-11-12T20:46:36.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42402025-11-12T20:46:36.432ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
42412025-11-12T20:46:36.432ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
42422025-11-12T20:46:36.432ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = Cockroach step id = Ensure
42432025-11-12T20:46:36.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42442025-11-12T20:46:36.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42452025-11-12T20:46:36.450ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
42462025-11-12T20:46:36.450ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8919f271-94ed-4bef-a737-06b4eca0122e step component = MgsUpdates step id = Ensure
42472025-11-12T20:46:36.450ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Dependency
42482025-11-12T20:46:36.450ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Dependency
42492025-11-12T20:46:36.450ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42502025-11-12T20:46:36.450ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42512025-11-12T20:46:36.451ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.17623904s iteration = 3 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4230,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":107463152,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":107463101,"secs":0}},"event_index":1,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107478581,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":104713900,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":104713860,"secs":0}},"event_index":2,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":212207172,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":84462236,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":84462186,"secs":0}},"event_index":3,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":296686038,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":250264446,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":250264396,"secs":0}},"event_index":4,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":546968685,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":331952080,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":331952000,"secs":0}},"event_index":5,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":878936676,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":141605735,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":141605695,"secs":0}},"event_index":6,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":20555911,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3890,"secs":0}},"event_index":7,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":20569011,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2570,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2530,"secs":0}},"event_index":8,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":20574381,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3120,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3090,"secs":0}},"event_index":9,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":20579031,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1880,"secs":0}},"event_index":10,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":20582451,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":31898446,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"warning","message":"failed to initialize single-node clickhouse database: Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"3daa0d98-8da6-441f-b165-5d8364784808\\", \\"content-length\\": \\"94\\", \\"date\\": \\"Wed, 12 Nov 2025 20:46:36 GMT\\"}; value: Error { error_code: None, message: \\"Service Unavailable\\", request_id: \\"3daa0d98-8da6-441f-b165-5d8364784808\\" }","metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":31898406,"secs":0}},"event_index":11,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":52484167,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":59806408,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":59806308,"secs":0}},"event_index":12,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":112299795,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":45033716,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":45033676,"secs":0}},"event_index":13,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":157351271,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":18478548,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":18478498,"secs":0}},"event_index":14,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":175838739,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5030,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4990,"secs":0}},"event_index":15,"execution_id":"aca2c0ae-7291-49d9-b6a9-6d7d28cf03ff","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":175857020,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
42522025-11-12T20:46:36.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
42532025-11-12T20:46:36.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42542025-11-12T20:46:36.452ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42552025-11-12T20:46:36.452ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42562025-11-12T20:46:36.455ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
42572025-11-12T20:46:36.455ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42582025-11-12T20:46:36.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
42592025-11-12T20:46:36.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42602025-11-12T20:46:36.457ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42612025-11-12T20:46:36.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42622025-11-12T20:46:36.493ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42632025-11-12T20:46:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42642025-11-12T20:46:36.494ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(af59bb51-5443-4af7-b670-a3bb2595f0bc, 0), (17edc7d5-82c6-45b6-8538-5f081fb1baff, 0), (1251654b-1450-4a6f-ae07-c188425653da, 0)]
42652025-11-12T20:46:36.494ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42662025-11-12T20:46:36.495ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 992422814 } })
42672025-11-12T20:46:36.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
42682025-11-12T20:46:36.497ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
42692025-11-12T20:46:36.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
42702025-11-12T20:46:36.499ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
42712025-11-12T20:46:36.499ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 48.981278ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
42722025-11-12T20:46:36.528ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 992, tv_nsec: 25958391 } })
42732025-11-12T20:46:36.529ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
42742025-11-12T20:46:36.529ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
42752025-11-12T20:46:36.529ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
42762025-11-12T20:46:36.529ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff588822-240a-4717-90bb-e38d582b84f5", "content-length": "476", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42772025-11-12T20:46:36.529ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
42782025-11-12T20:46:36.530ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "afb84081-f1b3-4d3a-a5b8-781ecaf1cac6", "content-length": "734", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42792025-11-12T20:46:36.530ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
42802025-11-12T20:46:36.530ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58b26dad-b559-44d0-b9e1-679b157c6a67", "content-length": "111", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42812025-11-12T20:46:36.530ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
42822025-11-12T20:46:36.531ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c89a197-e5a5-433d-a45e-3780f8634c80", "content-length": "111", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42832025-11-12T20:46:36.531ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
42842025-11-12T20:46:36.531ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6af833f0-2303-4f16-a076-6842f0fdfc14", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42852025-11-12T20:46:36.531ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
42862025-11-12T20:46:36.532ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "046e777b-7554-40a2-8061-ff71f114ca0f", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42872025-11-12T20:46:36.532ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
42882025-11-12T20:46:36.532ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8746044-8db8-4d55-8c8d-a9cbc8e54a2c", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42892025-11-12T20:46:36.532ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
42902025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc1eb321-79d8-459d-a216-388e093cfcc1", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42912025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
42922025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73a08d55-2e50-42b8-af78-c32bf3458ab7", "content-length": "702", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42932025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
42942025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3a9b9e0-936b-4705-9b38-dd20baf65af6", "content-length": "727", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42952025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
42962025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d28a5e3-b934-4792-9938-6e4ee8416dec", "content-length": "729", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42972025-11-12T20:46:36.533ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
42982025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85870df8-8f0b-453d-9405-f6868a191b9e", "content-length": "728", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
42992025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
43002025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17c841a4-4bde-4443-8b14-2ece8a4da09a", "content-length": "722", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43012025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
43022025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a564741-4176-4e4b-9214-2813489eeaa1", "content-length": "10", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43032025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
43042025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0573140e-668d-4d7e-a0f9-20e984053d90", "content-length": "144", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43052025-11-12T20:46:36.534ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
43062025-11-12T20:46:36.535ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "706c0e66-da6f-4fbe-b561-af8a93773a9e", "content-length": "147", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43072025-11-12T20:46:36.535ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
43082025-11-12T20:46:36.536ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dda3af1b-4b12-45b4-bf1a-f32c3904f1b3", "content-length": "109", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43092025-11-12T20:46:36.536ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
43102025-11-12T20:46:36.536ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "476de625-63f2-4fea-a553-1176cdd376d0", "content-length": "109", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43112025-11-12T20:46:36.536ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
43122025-11-12T20:46:36.537ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f3b4277-ed72-473e-9dd2-d5dc7cc51d74", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43132025-11-12T20:46:36.537ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
43142025-11-12T20:46:36.537ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "169dfd0e-6c68-43a6-9369-42853fe96a0b", "content-length": "163", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43152025-11-12T20:46:36.537ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
43162025-11-12T20:46:36.538ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa3dfb2c-a90d-4702-ab97-4bb5e8023c94", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43172025-11-12T20:46:36.538ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
43182025-11-12T20:46:36.538ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d3b511d-942a-4571-ad92-868454da94c9", "content-length": "166", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43192025-11-12T20:46:36.538ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
43202025-11-12T20:46:36.538ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bac43534-07b8-4780-8808-0b2d3de08a9e", "content-length": "702", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43212025-11-12T20:46:36.538ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
43222025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ebfdd79-5de6-4df9-870d-e2f3a713333c", "content-length": "727", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43232025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
43242025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d86e5307-8712-457e-974c-5d9160e69708", "content-length": "729", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43252025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
43262025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b17edb3-d3b0-4393-8691-1d91ea411b27", "content-length": "728", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43272025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43282025-11-12T20:46:36.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43292025-11-12T20:46:36.541ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43302025-11-12T20:46:36.543ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
43312025-11-12T20:46:36.543ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43322025-11-12T20:46:36.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43332025-11-12T20:46:36.545ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43342025-11-12T20:46:36.547ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
43352025-11-12T20:46:36.547ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43362025-11-12T20:46:36.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43372025-11-12T20:46:36.669ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
43382025-11-12T20:46:36.669ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
43392025-11-12T20:46:36.732ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8addfd1-d580-4d43-b425-3fbae4866972", "content-length": "95174", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43402025-11-12T20:46:36.736ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
43412025-11-12T20:46:36.736ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
43422025-11-12T20:46:36.736ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
43432025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
43442025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
43452025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
43462025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
43472025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
43482025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da8563bf-7c63-47f0-b8d2-03609a816b71", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43492025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
43502025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b60b5a91-593e-4242-86da-fd350710fb3d", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43512025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7b4e4ba-5d54-4159-9ff3-93995e82a3a5", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43522025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef0d5b98-85bb-411e-b0c1-6820cfb47fe0", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43532025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
43542025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
43552025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac7c8cec-5a55-4302-8003-a0a2ee7a8417", "content-length": "64", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43562025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
43572025-11-12T20:46:36.736ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
43582025-11-12T20:46:36.779ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ff8741-b826-4ae6-9e42-5df72eb9bd51", "content-length": "372323", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43592025-11-12T20:46:36.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "beb7479a-7eaa-4138-bde3-c9b0c738841a", "content-length": "369708", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43602025-11-12T20:46:36.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0ab9b9c-23c9-4e1f-acb7-d52caebd7188", "content-length": "410943", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43612025-11-12T20:46:36.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe17835b-857c-470b-844c-1d2240daf97a", "content-length": "373662", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43622025-11-12T20:46:36.781ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7bcde93-c994-4973-8e46-75f6fc23a52c", "content-length": "376398", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43632025-11-12T20:46:36.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
43642025-11-12T20:46:36.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
43652025-11-12T20:46:36.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
43662025-11-12T20:46:36.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
43672025-11-12T20:46:36.784ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
43682025-11-12T20:46:36.815ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
43692025-11-12T20:46:36.815ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
43702025-11-12T20:46:36.819ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "276fdbdd-247e-405e-83ef-16c120d83048", "content-length": "126", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43712025-11-12T20:46:36.820ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
43722025-11-12T20:46:36.911ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
43732025-11-12T20:46:36.911ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
43742025-11-12T20:46:36.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "39b64302-64e6-47a4-bf51-c2d0ab5fef6b", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43752025-11-12T20:46:36.911ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
43762025-11-12T20:46:36.912ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
43772025-11-12T20:46:36.912ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
43782025-11-12T20:46:36.913ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "23225e4d-02e7-4de6-a6e5-bc70aa87a254", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43792025-11-12T20:46:36.913ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
43802025-11-12T20:46:36.914ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
43812025-11-12T20:46:36.914ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
43822025-11-12T20:46:36.915ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "23990bf7-3167-415d-9035-5febab4b16b4", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:46:36 GMT"} })
43832025-11-12T20:46:36.915ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
43842025-11-12T20:46:36.916ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
43852025-11-12T20:46:36.916ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
43862025-11-12T20:46:36.918ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43872025-11-12T20:46:36.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43882025-11-12T20:46:36.919ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43892025-11-12T20:46:36.922ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
43902025-11-12T20:46:36.922ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43912025-11-12T20:46:36.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
43922025-11-12T20:46:37.123ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1b79856a-d0c9-4661-821f-cb877e3669c1 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
43932025-11-12T20:46:37.123ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1b79856a-d0c9-4661-821f-cb877e3669c1 time_started = 2025-11-12 20:46:36.529462 UTC
43942025-11-12T20:46:37.123ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 8 reason = Dependency
43952025-11-12T20:46:37.123ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43962025-11-12T20:46:37.123ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 673.097088ms iteration = 3 status = {"collection_id":"1b79856a-d0c9-4661-821f-cb877e3669c1","time_done":"2025-11-12 20:46:36.916661 UTC","time_started":"2025-11-12 20:46:36.529462 UTC"}
43972025-11-12T20:46:37.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43982025-11-12T20:46:37.126ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43992025-11-12T20:46:37.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
44002025-11-12T20:46:37.129ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
44012025-11-12T20:46:37.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
44022025-11-12T20:46:37.132ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
44032025-11-12T20:46:37.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
44042025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Dependency
44052025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 47.401097ms iteration = 8 status = {"Loaded":{"collection_id":"be6f941b-6853-4193-8c1b-4c65d2be0df1","time_loaded":"2025-11-12T20:46:37.123803051Z","time_started":"2025-11-12T20:46:36.587806Z"}}
44062025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 8 reason = Dependency
44072025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Dependency
44082025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44092025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44102025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44112025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44122025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44132025-11-12T20:46:37.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44142025-11-12T20:46:37.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44152025-11-12T20:46:37.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44162025-11-12T20:46:37.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44172025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
44182025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44192025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44202025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
44212025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44222025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
44232025-11-12T20:46:37.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44242025-11-12T20:46:37.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44252025-11-12T20:46:37.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44262025-11-12T20:46:37.178ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44272025-11-12T20:46:37.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44282025-11-12T20:46:37.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44292025-11-12T20:46:37.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44302025-11-12T20:46:37.179ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44312025-11-12T20:46:37.181ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
44322025-11-12T20:46:37.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44332025-11-12T20:46:37.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44342025-11-12T20:46:37.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
44352025-11-12T20:46:37.182ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44362025-11-12T20:46:37.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44372025-11-12T20:46:37.182ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
44382025-11-12T20:46:37.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44392025-11-12T20:46:37.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44402025-11-12T20:46:37.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44412025-11-12T20:46:37.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44422025-11-12T20:46:37.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44432025-11-12T20:46:37.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44442025-11-12T20:46:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44452025-11-12T20:46:37.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44462025-11-12T20:46:37.187ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
44472025-11-12T20:46:37.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44482025-11-12T20:46:37.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44492025-11-12T20:46:37.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44502025-11-12T20:46:37.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
44512025-11-12T20:46:37.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44522025-11-12T20:46:37.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44532025-11-12T20:46:37.191ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
44542025-11-12T20:46:37.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44552025-11-12T20:46:37.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44562025-11-12T20:46:37.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44572025-11-12T20:46:37.193ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
44582025-11-12T20:46:37.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44592025-11-12T20:46:37.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44602025-11-12T20:46:37.196ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
44612025-11-12T20:46:37.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44622025-11-12T20:46:37.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44632025-11-12T20:46:37.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44642025-11-12T20:46:37.201ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
44652025-11-12T20:46:37.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44662025-11-12T20:46:37.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44672025-11-12T20:46:37.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44682025-11-12T20:46:37.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
44692025-11-12T20:46:37.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
44702025-11-12T20:46:37.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
44712025-11-12T20:46:37.209ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
44722025-11-12T20:46:37.209ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 38.184579ms iteration = 5 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"be6f941b-6853-4193-8c1b-4c65d2be0df1","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
44732025-11-12T20:46:37.234ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 62.817575ms iteration = 7 status = {"physical_disks_added":0}
44742025-11-12T20:46:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44752025-11-12T20:46:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44762025-11-12T20:46:37.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44772025-11-12T20:46:37.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44782025-11-12T20:46:37.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44792025-11-12T20:46:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44802025-11-12T20:46:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44812025-11-12T20:46:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44822025-11-12T20:46:37.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44832025-11-12T20:46:37.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44842025-11-12T20:46:37.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44852025-11-12T20:46:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
44862025-11-12T20:46:37.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44872025-11-12T20:46:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44882025-11-12T20:46:37.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44892025-11-12T20:46:37.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44902025-11-12T20:46:37.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44912025-11-12T20:46:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44922025-11-12T20:46:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44932025-11-12T20:46:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44942025-11-12T20:46:37.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44952025-11-12T20:46:37.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
44962025-11-12T20:46:37.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44972025-11-12T20:46:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44982025-11-12T20:46:37.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44992025-11-12T20:46:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45002025-11-12T20:46:37.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45012025-11-12T20:46:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
45022025-11-12T20:46:37.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45032025-11-12T20:46:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45042025-11-12T20:46:37.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45052025-11-12T20:46:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45062025-11-12T20:46:37.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45072025-11-12T20:46:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
45082025-11-12T20:46:37.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45092025-11-12T20:46:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45102025-11-12T20:46:37.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45112025-11-12T20:46:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45122025-11-12T20:46:37.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45132025-11-12T20:46:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45142025-11-12T20:46:37.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45152025-11-12T20:46:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
45162025-11-12T20:46:37.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45172025-11-12T20:46:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45182025-11-12T20:46:37.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45192025-11-12T20:46:37.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
45202025-11-12T20:46:37.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45212025-11-12T20:46:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45222025-11-12T20:46:37.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45232025-11-12T20:46:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45242025-11-12T20:46:37.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45252025-11-12T20:46:37.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45262025-11-12T20:46:37.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45272025-11-12T20:46:37.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
45282025-11-12T20:46:37.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45292025-11-12T20:46:37.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45302025-11-12T20:46:37.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45312025-11-12T20:46:37.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
45322025-11-12T20:46:37.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45332025-11-12T20:46:37.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45342025-11-12T20:46:37.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45352025-11-12T20:46:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45362025-11-12T20:46:37.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45372025-11-12T20:46:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45382025-11-12T20:46:37.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45392025-11-12T20:46:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
45402025-11-12T20:46:37.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45412025-11-12T20:46:37.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45422025-11-12T20:46:37.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45432025-11-12T20:46:37.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45442025-11-12T20:46:37.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45452025-11-12T20:46:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
45462025-11-12T20:46:37.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45472025-11-12T20:46:37.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45482025-11-12T20:46:37.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45492025-11-12T20:46:37.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45502025-11-12T20:46:37.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45512025-11-12T20:46:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
45522025-11-12T20:46:37.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45532025-11-12T20:46:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45542025-11-12T20:46:37.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45552025-11-12T20:46:37.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45562025-11-12T20:46:37.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45572025-11-12T20:46:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
45582025-11-12T20:46:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45592025-11-12T20:46:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45602025-11-12T20:46:37.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45612025-11-12T20:46:37.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45622025-11-12T20:46:37.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45632025-11-12T20:46:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
45642025-11-12T20:46:37.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45652025-11-12T20:46:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45662025-11-12T20:46:37.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45672025-11-12T20:46:37.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45682025-11-12T20:46:37.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45692025-11-12T20:46:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
45702025-11-12T20:46:37.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45712025-11-12T20:46:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45722025-11-12T20:46:37.401ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
45732025-11-12T20:46:37.401ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
45742025-11-12T20:46:37.401ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = ea011b61-2676-4a34-822c-7b1e1c34090c parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
45752025-11-12T20:46:37.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45762025-11-12T20:46:37.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45772025-11-12T20:46:37.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45782025-11-12T20:46:37.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
45792025-11-12T20:46:37.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45802025-11-12T20:46:37.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45812025-11-12T20:46:37.409ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
45822025-11-12T20:46:37.409ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
45832025-11-12T20:46:37.409ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 238.491752ms iteration = 8 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
45842025-11-12T20:46:40.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 14 reason = Timeout
45852025-11-12T20:46:40.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
45862025-11-12T20:46:40.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
45872025-11-12T20:46:40.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45882025-11-12T20:46:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
45892025-11-12T20:46:40.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
45902025-11-12T20:46:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
45912025-11-12T20:46:40.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
45922025-11-12T20:46:40.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.654652ms iteration = 14 status = {"config_updated":false}
45932025-11-12T20:46:45.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
45942025-11-12T20:46:45.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
45952025-11-12T20:46:45.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45962025-11-12T20:46:45.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 15 reason = Timeout
45972025-11-12T20:46:45.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45982025-11-12T20:46:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
45992025-11-12T20:46:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46002025-11-12T20:46:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
46012025-11-12T20:46:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46022025-11-12T20:46:45.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46032025-11-12T20:46:45.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46042025-11-12T20:46:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46052025-11-12T20:46:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46062025-11-12T20:46:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
46072025-11-12T20:46:45.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
46082025-11-12T20:46:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
46092025-11-12T20:46:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46102025-11-12T20:46:45.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46112025-11-12T20:46:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46122025-11-12T20:46:45.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
46132025-11-12T20:46:45.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.932839ms iteration = 15 status = {"config_updated":false}
46142025-11-12T20:46:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46152025-11-12T20:46:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46162025-11-12T20:46:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46172025-11-12T20:46:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
46182025-11-12T20:46:45.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46192025-11-12T20:46:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46202025-11-12T20:46:45.305ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
46212025-11-12T20:46:45.305ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.455761ms iteration = 8 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
46222025-11-12T20:46:45.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.13063ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46232025-11-12T20:46:47.914ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:63649
46242025-11-12T20:46:47.914ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 8437e937-0d02-438c-8797-313c8dc9cbb4 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
46252025-11-12T20:46:47.920ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 8437e937-0d02-438c-8797-313c8dc9cbb4 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
46262025-11-12T20:46:47.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 8437e937-0d02-438c-8797-313c8dc9cbb4 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
46272025-11-12T20:46:47.925ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 11211 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 8437e937-0d02-438c-8797-313c8dc9cbb4 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
46282025-11-12T20:46:47.925ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 26a2147c-12b7-4585-801b-75486350402b uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
46292025-11-12T20:46:47.928ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 26a2147c-12b7-4585-801b-75486350402b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
46302025-11-12T20:46:47.928ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 26a2147c-12b7-4585-801b-75486350402b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
46312025-11-12T20:46:47.935ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 9962 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 26a2147c-12b7-4585-801b-75486350402b response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
46322025-11-12T20:46:47.998ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:46805 remote_addr = [fd00:1122:3344:101::d]:38173
46332025-11-12T20:46:48.008ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 9542 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 5365f6fa-592c-4334-ab76-501c0b93122b response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
46342025-11-12T20:46:50.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 9 reason = Timeout
46352025-11-12T20:46:50.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
46362025-11-12T20:46:50.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
46372025-11-12T20:46:50.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 16 reason = Timeout
46382025-11-12T20:46:50.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
46392025-11-12T20:46:50.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
46402025-11-12T20:46:50.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46412025-11-12T20:46:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46422025-11-12T20:46:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
46432025-11-12T20:46:50.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
46442025-11-12T20:46:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
46452025-11-12T20:46:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
46462025-11-12T20:46:50.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
46472025-11-12T20:46:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
46482025-11-12T20:46:50.283ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = be6f941b-6853-4193-8c1b-4c65d2be0df1 original_time_started = 2025-11-12 20:46:36.587806 UTC
46492025-11-12T20:46:50.283ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.095984ms iteration = 9 status = {"Loaded":{"collection_id":"be6f941b-6853-4193-8c1b-4c65d2be0df1","time_loaded":"2025-11-12T20:46:50.276797417Z","time_started":"2025-11-12T20:46:36.587806Z"}}
46502025-11-12T20:46:50.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 6 reason = Timeout
46512025-11-12T20:46:50.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
46522025-11-12T20:46:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
46532025-11-12T20:46:50.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
46542025-11-12T20:46:50.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.708159ms iteration = 16 status = {"config_updated":false}
46552025-11-12T20:46:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46562025-11-12T20:46:50.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
46572025-11-12T20:46:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
46582025-11-12T20:46:50.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
46592025-11-12T20:46:50.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.407679ms iteration = 6 status = "NoSitrep"
46602025-11-12T20:46:50.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 6 reason = Timeout
46612025-11-12T20:46:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
46622025-11-12T20:46:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
46632025-11-12T20:46:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
46642025-11-12T20:46:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
46652025-11-12T20:46:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46662025-11-12T20:46:50.294ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
46672025-11-12T20:46:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
46682025-11-12T20:46:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
46692025-11-12T20:46:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
46702025-11-12T20:46:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
46712025-11-12T20:46:50.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46722025-11-12T20:46:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
46732025-11-12T20:46:50.300ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
46742025-11-12T20:46:50.300ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.478321ms iteration = 6 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
46752025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
46762025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
46772025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46782025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46792025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 17 reason = Timeout
46802025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
46812025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46822025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46832025-11-12T20:46:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
46842025-11-12T20:46:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46852025-11-12T20:46:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46862025-11-12T20:46:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46872025-11-12T20:46:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46882025-11-12T20:46:55.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.132777ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46892025-11-12T20:46:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46902025-11-12T20:46:55.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46912025-11-12T20:46:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
46922025-11-12T20:46:55.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
46932025-11-12T20:46:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46942025-11-12T20:46:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
46952025-11-12T20:46:55.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46962025-11-12T20:46:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46972025-11-12T20:46:55.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
46982025-11-12T20:46:55.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.015218ms iteration = 17 status = {"config_updated":false}
46992025-11-12T20:46:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47002025-11-12T20:46:55.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
47012025-11-12T20:46:55.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47022025-11-12T20:46:55.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
47032025-11-12T20:46:55.300ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
47042025-11-12T20:46:55.300ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.536199ms iteration = 9 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
47052025-11-12T20:46:57.941ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 331 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = eb1dee94-bc76-4f82-8b30-7be81c3d970a response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
47062025-11-12T20:47:00.282ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 18 reason = Timeout
47072025-11-12T20:47:00.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47082025-11-12T20:47:00.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47092025-11-12T20:47:00.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47102025-11-12T20:47:00.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
47112025-11-12T20:47:00.287ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47122025-11-12T20:47:00.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47132025-11-12T20:47:00.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
47142025-11-12T20:47:00.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.432596ms iteration = 18 status = {"config_updated":false}
47152025-11-12T20:47:05.277ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
47162025-11-12T20:47:05.277ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 4 reason = Timeout
47172025-11-12T20:47:05.277ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
47182025-11-12T20:47:05.277ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
47192025-11-12T20:47:05.277ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
47202025-11-12T20:47:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
47212025-11-12T20:47:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
47222025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 10 reason = Timeout
47232025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
47242025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
47252025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
47262025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
47272025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
47282025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
47292025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47302025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47312025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
47322025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47332025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
47342025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
47352025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47362025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
47372025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47382025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 4 reason = Timeout
47392025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 19 reason = Timeout
47402025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
47412025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
47422025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47432025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
47442025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47452025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
47462025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
47472025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
47482025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47492025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
47502025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
47512025-11-12T20:47:05.280ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
47522025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
47532025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
47542025-11-12T20:47:05.281ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
47552025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47562025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
47572025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
47582025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47592025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
47602025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
47612025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47622025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
47632025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
47642025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47652025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
47662025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
47672025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
47682025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47692025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47702025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
47712025-11-12T20:47:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1020, tv_nsec: 775107380 } })
47722025-11-12T20:47:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1020, tv_nsec: 775132101 } })
47732025-11-12T20:47:05.281ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1020, tv_nsec: 775132561 } })
47742025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
47752025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
47762025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47772025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47782025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
47792025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
47802025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47812025-11-12T20:47:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
47822025-11-12T20:47:05.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
47832025-11-12T20:47:05.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47842025-11-12T20:47:05.283ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47852025-11-12T20:47:05.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47862025-11-12T20:47:05.283ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47872025-11-12T20:47:05.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.145361ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
47882025-11-12T20:47:05.284ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47892025-11-12T20:47:05.284ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.522899ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
47902025-11-12T20:47:05.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47912025-11-12T20:47:05.285ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47922025-11-12T20:47:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47932025-11-12T20:47:05.288ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 7 reason = Timeout
47942025-11-12T20:47:05.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
47952025-11-12T20:47:05.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47962025-11-12T20:47:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
47972025-11-12T20:47:05.289ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
47982025-11-12T20:47:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47992025-11-12T20:47:05.289ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48002025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48012025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48022025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
48032025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48042025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48052025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
48062025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
48072025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48082025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
48092025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48102025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48112025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
48122025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
48132025-11-12T20:47:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48142025-11-12T20:47:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48152025-11-12T20:47:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 11.360705ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
48162025-11-12T20:47:05.292ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = be6f941b-6853-4193-8c1b-4c65d2be0df1 original_time_started = 2025-11-12 20:46:36.587806 UTC
48172025-11-12T20:47:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.524684ms iteration = 10 status = {"Loaded":{"collection_id":"be6f941b-6853-4193-8c1b-4c65d2be0df1","time_loaded":"2025-11-12T20:47:05.280001116Z","time_started":"2025-11-12T20:46:36.587806Z"}}
48182025-11-12T20:47:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
48192025-11-12T20:47:05.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48202025-11-12T20:47:05.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48212025-11-12T20:47:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48222025-11-12T20:47:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48232025-11-12T20:47:05.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 7 reason = Timeout
48242025-11-12T20:47:05.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
48252025-11-12T20:47:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
48262025-11-12T20:47:05.294ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1020, tv_nsec: 787676915 } })
48272025-11-12T20:47:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
48282025-11-12T20:47:05.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48292025-11-12T20:47:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
48302025-11-12T20:47:05.294ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
48312025-11-12T20:47:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48322025-11-12T20:47:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
48332025-11-12T20:47:05.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48342025-11-12T20:47:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48352025-11-12T20:47:05.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
48362025-11-12T20:47:05.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
48372025-11-12T20:47:05.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48382025-11-12T20:47:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
48392025-11-12T20:47:05.297ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.756684ms iteration = 4 status = 0
48402025-11-12T20:47:05.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48412025-11-12T20:47:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
48422025-11-12T20:47:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
48432025-11-12T20:47:05.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48442025-11-12T20:47:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
48452025-11-12T20:47:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 15.901932ms iteration = 7 status = "NoSitrep"
48462025-11-12T20:47:05.310ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
48472025-11-12T20:47:05.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
48482025-11-12T20:47:05.313ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
48492025-11-12T20:47:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
48502025-11-12T20:47:05.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48512025-11-12T20:47:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
48522025-11-12T20:47:05.320ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
48532025-11-12T20:47:05.320ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 27.115841ms iteration = 7 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
48542025-11-12T20:47:05.356ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48552025-11-12T20:47:05.357ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48562025-11-12T20:47:05.357ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
48572025-11-12T20:47:05.357ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48582025-11-12T20:47:05.357ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
48592025-11-12T20:47:05.357ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.842059ms iteration = 10 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
48602025-11-12T20:47:05.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
48612025-11-12T20:47:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
48622025-11-12T20:47:05.362ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
48632025-11-12T20:47:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
48642025-11-12T20:47:05.363ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
48652025-11-12T20:47:05.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
48662025-11-12T20:47:05.364ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48672025-11-12T20:47:05.364ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
48682025-11-12T20:47:05.364ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 83.521961ms iteration = 19 status = {"config_updated":false}
48692025-11-12T20:47:05.368ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48702025-11-12T20:47:05.368ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48712025-11-12T20:47:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48722025-11-12T20:47:05.369ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 88.954792ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
48732025-11-12T20:47:05.370ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48742025-11-12T20:47:05.371ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 90.595824ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
48752025-11-12T20:47:05.374ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48762025-11-12T20:47:05.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
48772025-11-12T20:47:05.374ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48782025-11-12T20:47:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48792025-11-12T20:47:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48802025-11-12T20:47:05.375ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48812025-11-12T20:47:05.378ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48822025-11-12T20:47:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48832025-11-12T20:47:05.379ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48842025-11-12T20:47:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48852025-11-12T20:47:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48862025-11-12T20:47:05.379ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48872025-11-12T20:47:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48882025-11-12T20:47:05.380ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
48892025-11-12T20:47:05.380ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 99.553917ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
48902025-11-12T20:47:05.381ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48912025-11-12T20:47:05.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
48922025-11-12T20:47:05.384ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
48932025-11-12T20:47:05.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
48942025-11-12T20:47:05.398ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 117.703436ms iteration = 8 status = {"physical_disks_added":0}
48952025-11-12T20:47:05.420ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
48962025-11-12T20:47:05.421ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
48972025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "9063c41a-950a-4a7f-b238-7f6f6ea76e26", "content-length": "78", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
48982025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
48992025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
49002025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
49012025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 144.632943ms iteration = 4 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
49022025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
49032025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
49042025-11-12T20:47:05.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
49052025-11-12T20:47:05.423ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 143.0043ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49062025-11-12T20:47:05.424ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
49072025-11-12T20:47:05.424ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49082025-11-12T20:47:05.424ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
49092025-11-12T20:47:05.424ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
49102025-11-12T20:47:05.424ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b4a3777-79f6-47ea-8184-e415eaac8148", "content-length": "26", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49112025-11-12T20:47:05.424ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49122025-11-12T20:47:05.425ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
49132025-11-12T20:47:05.426ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
49142025-11-12T20:47:05.426ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
49152025-11-12T20:47:05.426ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49162025-11-12T20:47:05.427ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb0a22c2-43bf-4c41-b809-5f03b6738938", "content-length": "26", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49172025-11-12T20:47:05.427ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49182025-11-12T20:47:05.428ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
49192025-11-12T20:47:05.428ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
49202025-11-12T20:47:05.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
49212025-11-12T20:47:05.429ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
49222025-11-12T20:47:05.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
49232025-11-12T20:47:05.430ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
49242025-11-12T20:47:05.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
49252025-11-12T20:47:05.430ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
49262025-11-12T20:47:05.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
49272025-11-12T20:47:05.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
49282025-11-12T20:47:05.431ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49292025-11-12T20:47:05.431ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
49302025-11-12T20:47:05.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49312025-11-12T20:47:05.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
49322025-11-12T20:47:05.435ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
49332025-11-12T20:47:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
49342025-11-12T20:47:05.444ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 163.444009ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49352025-11-12T20:47:05.445ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
49362025-11-12T20:47:05.445ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 164.922788ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
49372025-11-12T20:47:05.488ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
49382025-11-12T20:47:05.488ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
49392025-11-12T20:47:05.489ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c9f4c2c-a2c7-4a9c-9b0c-bb59a7763a1e", "content-length": "81", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49402025-11-12T20:47:05.490ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
49412025-11-12T20:47:05.490ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
49422025-11-12T20:47:05.490ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "109b83ab-657b-4291-86db-1deff1217bc5", "content-length": "81", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49432025-11-12T20:47:05.492ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
49442025-11-12T20:47:05.492ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
49452025-11-12T20:47:05.492ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "822a2ffe-32ff-459a-b975-66d3d9c40822", "content-length": "81", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49462025-11-12T20:47:05.493ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49472025-11-12T20:47:05.493ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
49482025-11-12T20:47:05.493ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 212.686034ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
49492025-11-12T20:47:05.494ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 213.963537ms iteration = 4 status = {"disabled":false,"errors":[],"sps":[]}
49502025-11-12T20:47:05.496ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49512025-11-12T20:47:05.496ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
49522025-11-12T20:47:05.496ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49532025-11-12T20:47:05.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
49542025-11-12T20:47:05.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 216.472632ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
49552025-11-12T20:47:05.505ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "532fe32d-720b-476f-9551-244b107b0d1c", "content-length": "372", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49562025-11-12T20:47:05.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49572025-11-12T20:47:05.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49582025-11-12T20:47:05.519ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 12809061 } })
49592025-11-12T20:47:05.520ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 242.518516ms iteration = 4 status = {}
49602025-11-12T20:47:05.586ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
49612025-11-12T20:47:05.586ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
49622025-11-12T20:47:05.587ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "49252887-4cb5-4b58-8a92-31a5fc7a76f6", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49632025-11-12T20:47:05.587ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49642025-11-12T20:47:05.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49652025-11-12T20:47:05.590ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49662025-11-12T20:47:05.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49672025-11-12T20:47:05.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49682025-11-12T20:47:05.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49692025-11-12T20:47:05.594ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49702025-11-12T20:47:05.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49712025-11-12T20:47:05.596ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
49722025-11-12T20:47:05.596ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49732025-11-12T20:47:05.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49742025-11-12T20:47:05.601ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
49752025-11-12T20:47:05.601ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49762025-11-12T20:47:05.602ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "88a2eb49-57ac-4a1b-85c9-6e229e5f63da", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
49772025-11-12T20:47:05.605ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 324.383555ms iteration = 6 status = {}
49782025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
49792025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
49802025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
49812025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
49822025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49832025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49842025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49852025-11-12T20:47:05.614ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49862025-11-12T20:47:05.615ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
49872025-11-12T20:47:05.615ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
49882025-11-12T20:47:05.615ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
49892025-11-12T20:47:05.615ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49902025-11-12T20:47:05.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49912025-11-12T20:47:05.618ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
49922025-11-12T20:47:05.618ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
49932025-11-12T20:47:05.647ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49942025-11-12T20:47:05.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
49952025-11-12T20:47:05.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49962025-11-12T20:47:05.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
49972025-11-12T20:47:05.696ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
49982025-11-12T20:47:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
49992025-11-12T20:47:05.738ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50002025-11-12T20:47:05.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
50012025-11-12T20:47:05.741ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
50022025-11-12T20:47:05.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
50032025-11-12T20:47:05.775ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e2cfc2f8-cb67-4d55-a8c8-4a1c34992cc5", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
50042025-11-12T20:47:05.775ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
50052025-11-12T20:47:05.775ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
50062025-11-12T20:47:05.782ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa324421-c364-4342-9d11-ae23c098f905", "content-length": "682", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
50072025-11-12T20:47:05.782ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50082025-11-12T20:47:05.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50092025-11-12T20:47:05.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50102025-11-12T20:47:05.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50112025-11-12T20:47:05.798ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50122025-11-12T20:47:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50132025-11-12T20:47:05.801ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50142025-11-12T20:47:05.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50152025-11-12T20:47:05.809ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 528.729483ms iteration = 4 status = {}
50162025-11-12T20:47:05.873ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
50172025-11-12T20:47:05.874ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e425ff34-1236-4910-aa71-fd14babda82a", "content-length": "691", "date": "Wed, 12 Nov 2025 20:47:05 GMT"} })
50182025-11-12T20:47:05.874ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
50192025-11-12T20:47:05.874ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
50202025-11-12T20:47:05.874ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
50212025-11-12T20:47:05.876ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 595.890941ms iteration = 4 status = {}
50222025-11-12T20:47:07.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 361 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 230a3e2c-b0da-4fbe-9583-1d782a1b319e response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
50232025-11-12T20:47:10.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 20 reason = Timeout
50242025-11-12T20:47:10.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
50252025-11-12T20:47:10.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
50262025-11-12T20:47:10.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50272025-11-12T20:47:10.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
50282025-11-12T20:47:10.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
50292025-11-12T20:47:10.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
50302025-11-12T20:47:10.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
50312025-11-12T20:47:10.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.70854ms iteration = 20 status = {"config_updated":false}
50322025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
50332025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
50342025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50352025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50362025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 21 reason = Timeout
50372025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
50382025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50392025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50402025-11-12T20:47:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
50412025-11-12T20:47:15.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50422025-11-12T20:47:15.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50432025-11-12T20:47:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
50442025-11-12T20:47:15.287ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
50452025-11-12T20:47:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
50462025-11-12T20:47:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
50472025-11-12T20:47:15.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50482025-11-12T20:47:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50492025-11-12T20:47:15.289ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
50502025-11-12T20:47:15.289ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.463259ms iteration = 21 status = {"config_updated":false}
50512025-11-12T20:47:15.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50522025-11-12T20:47:15.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50532025-11-12T20:47:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
50542025-11-12T20:47:15.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
50552025-11-12T20:47:15.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50562025-11-12T20:47:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
50572025-11-12T20:47:15.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
50582025-11-12T20:47:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
50592025-11-12T20:47:15.355ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
50602025-11-12T20:47:15.355ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 74.086677ms iteration = 11 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
50612025-11-12T20:47:15.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.253329ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
50622025-11-12T20:47:17.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = d697a856-855b-446e-98ca-697ee628bc2c uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
50632025-11-12T20:47:17.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = d697a856-855b-446e-98ca-697ee628bc2c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
50642025-11-12T20:47:17.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = d697a856-855b-446e-98ca-697ee628bc2c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
50652025-11-12T20:47:17.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3595 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = d697a856-855b-446e-98ca-697ee628bc2c response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
50662025-11-12T20:47:17.921ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 34bc6cc4-287e-43c5-8c64-e2da990b5dd0 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
50672025-11-12T20:47:17.922ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 34bc6cc4-287e-43c5-8c64-e2da990b5dd0 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
50682025-11-12T20:47:17.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 34bc6cc4-287e-43c5-8c64-e2da990b5dd0 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
50692025-11-12T20:47:17.924ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2758 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63649 req_id = 34bc6cc4-287e-43c5-8c64-e2da990b5dd0 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
50702025-11-12T20:47:17.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = c5ed5cca-6723-4ab9-95cd-ae488dff835d response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
50712025-11-12T20:47:20.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 11 reason = Timeout
50722025-11-12T20:47:20.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
50732025-11-12T20:47:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
50742025-11-12T20:47:20.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 22 reason = Timeout
50752025-11-12T20:47:20.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
50762025-11-12T20:47:20.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50772025-11-12T20:47:20.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
50782025-11-12T20:47:20.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50792025-11-12T20:47:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
50802025-11-12T20:47:20.284ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
50812025-11-12T20:47:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
50822025-11-12T20:47:20.286ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = be6f941b-6853-4193-8c1b-4c65d2be0df1 original_time_started = 2025-11-12 20:46:36.587806 UTC
50832025-11-12T20:47:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 6.977641ms iteration = 11 status = {"Loaded":{"collection_id":"be6f941b-6853-4193-8c1b-4c65d2be0df1","time_loaded":"2025-11-12T20:47:20.279635420Z","time_started":"2025-11-12T20:46:36.587806Z"}}
50842025-11-12T20:47:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
50852025-11-12T20:47:20.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
50862025-11-12T20:47:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
50872025-11-12T20:47:20.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 8 reason = Timeout
50882025-11-12T20:47:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
50892025-11-12T20:47:20.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
50902025-11-12T20:47:20.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.072893ms iteration = 22 status = {"config_updated":false}
50912025-11-12T20:47:20.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
50922025-11-12T20:47:20.294ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 8 reason = Timeout
50932025-11-12T20:47:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
50942025-11-12T20:47:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
50952025-11-12T20:47:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
50962025-11-12T20:47:20.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
50972025-11-12T20:47:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50982025-11-12T20:47:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
50992025-11-12T20:47:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
51002025-11-12T20:47:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
51012025-11-12T20:47:20.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
51022025-11-12T20:47:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
51032025-11-12T20:47:20.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51042025-11-12T20:47:20.303ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
51052025-11-12T20:47:20.303ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
51062025-11-12T20:47:20.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.592949ms iteration = 8 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
51072025-11-12T20:47:20.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51082025-11-12T20:47:20.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
51092025-11-12T20:47:20.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
51102025-11-12T20:47:20.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
51112025-11-12T20:47:20.328ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 40.904569ms iteration = 8 status = "NoSitrep"
51122025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
51132025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
51142025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51152025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51162025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 23 reason = Timeout
51172025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51182025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51192025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51202025-11-12T20:47:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51212025-11-12T20:47:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51222025-11-12T20:47:25.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51232025-11-12T20:47:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51242025-11-12T20:47:25.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51252025-11-12T20:47:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51262025-11-12T20:47:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51272025-11-12T20:47:25.287ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51282025-11-12T20:47:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51292025-11-12T20:47:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51302025-11-12T20:47:25.289ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
51312025-11-12T20:47:25.289ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.947265ms iteration = 23 status = {"config_updated":false}
51322025-11-12T20:47:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51332025-11-12T20:47:25.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51342025-11-12T20:47:25.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
51352025-11-12T20:47:25.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51362025-11-12T20:47:25.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51372025-11-12T20:47:25.312ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
51382025-11-12T20:47:25.312ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.600347ms iteration = 12 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
51392025-11-12T20:47:25.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51402025-11-12T20:47:25.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51412025-11-12T20:47:25.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.791198ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51422025-11-12T20:47:27.941ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 370 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 3c612766-4439-4696-9d43-841d5742acfb response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
51432025-11-12T20:47:30.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 24 reason = Timeout
51442025-11-12T20:47:30.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51452025-11-12T20:47:30.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51462025-11-12T20:47:30.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51472025-11-12T20:47:30.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51482025-11-12T20:47:30.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51492025-11-12T20:47:30.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51502025-11-12T20:47:30.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
51512025-11-12T20:47:30.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.698523ms iteration = 24 status = {"config_updated":false}
51522025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
51532025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
51542025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
51552025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51562025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
51572025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
51582025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
51592025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
51602025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
51612025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
51622025-11-12T20:47:35.278ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
51632025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 5 reason = Timeout
51642025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
51652025-11-12T20:47:35.278ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = ExternalNetworking step id = Ensure
51662025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51672025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
51682025-11-12T20:47:35.278ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:37:35.278214067 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
51692025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
51702025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
51712025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
51722025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
51732025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51742025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
51752025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
51762025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
51772025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
51782025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51792025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
51802025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
51812025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
51822025-11-12T20:47:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51832025-11-12T20:47:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1050, tv_nsec: 772588642 } })
51842025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51852025-11-12T20:47:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1050, tv_nsec: 772688614 } })
51862025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 12 reason = Timeout
51872025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
51882025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
51892025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.386676ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
51902025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
51912025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.742705ms iteration = 3 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
51922025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
51932025-11-12T20:47:35.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
51942025-11-12T20:47:35.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
51952025-11-12T20:47:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
51962025-11-12T20:47:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
51972025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 3 reason = Timeout
51982025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
51992025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
52002025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52012025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 9 reason = Timeout
52022025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
52032025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52042025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52052025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
52062025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52072025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
52082025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52092025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
52102025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
52112025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52122025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52132025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 25 reason = Timeout
52142025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52152025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
52162025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52172025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
52182025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52192025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
52202025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
52212025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52222025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 5 reason = Timeout
52232025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
52242025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
52252025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
52262025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52272025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
52282025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
52292025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
52302025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
52312025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
52322025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52332025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
52342025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
52352025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
52362025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
52372025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 3 reason = Timeout
52382025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
52392025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
52402025-11-12T20:47:35.281ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
52412025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52422025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
52432025-11-12T20:47:35.281ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
52442025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52452025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
52462025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
52472025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
52482025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52492025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52502025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
52512025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52522025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52532025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
52542025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
52552025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52562025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52572025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52582025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52592025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52602025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
52612025-11-12T20:47:35.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1050, tv_nsec: 775576119 } })
52622025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
52632025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
52642025-11-12T20:47:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
52652025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
52662025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
52672025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52682025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
52692025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
52702025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52712025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52722025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
52732025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52742025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52752025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52762025-11-12T20:47:35.282ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1050, tv_nsec: 776284967 } })
52772025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52782025-11-12T20:47:35.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52792025-11-12T20:47:35.283ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
52802025-11-12T20:47:35.283ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:37:35.278214067 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
52812025-11-12T20:47:35.283ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.020639ms iteration = 3 status = {"expiration":"2025-11-12T20:37:35.278214067Z","pruned":[]}
52822025-11-12T20:47:35.284ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52832025-11-12T20:47:35.284ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1050, tv_nsec: 777801286 } })
52842025-11-12T20:47:35.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52852025-11-12T20:47:35.285ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52862025-11-12T20:47:35.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52872025-11-12T20:47:35.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
52882025-11-12T20:47:35.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
52892025-11-12T20:47:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
52902025-11-12T20:47:35.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 9 reason = Timeout
52912025-11-12T20:47:35.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
52922025-11-12T20:47:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
52932025-11-12T20:47:35.288ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.178593ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
52942025-11-12T20:47:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
52952025-11-12T20:47:35.289ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
52962025-11-12T20:47:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
52972025-11-12T20:47:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
52982025-11-12T20:47:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
52992025-11-12T20:47:35.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53002025-11-12T20:47:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53012025-11-12T20:47:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
53022025-11-12T20:47:35.291ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
53032025-11-12T20:47:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
53042025-11-12T20:47:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
53052025-11-12T20:47:35.292ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53062025-11-12T20:47:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53072025-11-12T20:47:35.293ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 3 reason = Timeout
53082025-11-12T20:47:35.293ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
53092025-11-12T20:47:35.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 9 reason = Timeout
53102025-11-12T20:47:35.293ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
53112025-11-12T20:47:35.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
53122025-11-12T20:47:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
53132025-11-12T20:47:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
53142025-11-12T20:47:35.378ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53152025-11-12T20:47:35.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
53162025-11-12T20:47:35.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
53172025-11-12T20:47:35.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
53182025-11-12T20:47:35.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 110.260984ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
53192025-11-12T20:47:35.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53202025-11-12T20:47:35.391ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53212025-11-12T20:47:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53222025-11-12T20:47:35.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53232025-11-12T20:47:35.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53242025-11-12T20:47:35.393ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53252025-11-12T20:47:35.396ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53262025-11-12T20:47:35.396ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53272025-11-12T20:47:35.397ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53282025-11-12T20:47:35.397ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 115.803726ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
53292025-11-12T20:47:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
53302025-11-12T20:47:35.397ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53312025-11-12T20:47:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53322025-11-12T20:47:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
53332025-11-12T20:47:35.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53342025-11-12T20:47:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53352025-11-12T20:47:35.399ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53362025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53372025-11-12T20:47:35.400ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
53382025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 119.075251ms iteration = 3 status = {"by_rx":{},"error":null}
53392025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
53402025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
53412025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
53422025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
53432025-11-12T20:47:35.400ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53442025-11-12T20:47:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53452025-11-12T20:47:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
53462025-11-12T20:47:35.401ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53472025-11-12T20:47:35.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 120.416516ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
53482025-11-12T20:47:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53492025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
53502025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53512025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
53522025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53532025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53542025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
53552025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53562025-11-12T20:47:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
53572025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
53582025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53592025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53602025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
53612025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53622025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
53632025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
53642025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53652025-11-12T20:47:35.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53662025-11-12T20:47:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53672025-11-12T20:47:35.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
53682025-11-12T20:47:35.404ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
53692025-11-12T20:47:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
53702025-11-12T20:47:35.404ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53712025-11-12T20:47:35.405ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53722025-11-12T20:47:35.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53732025-11-12T20:47:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53742025-11-12T20:47:35.408ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
53752025-11-12T20:47:35.408ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
53762025-11-12T20:47:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
53772025-11-12T20:47:35.409ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53782025-11-12T20:47:35.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
53792025-11-12T20:47:35.410ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = be6f941b-6853-4193-8c1b-4c65d2be0df1 original_time_started = 2025-11-12 20:46:36.587806 UTC
53802025-11-12T20:47:35.410ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 131.132104ms iteration = 12 status = {"Loaded":{"collection_id":"be6f941b-6853-4193-8c1b-4c65d2be0df1","time_loaded":"2025-11-12T20:47:35.279452349Z","time_started":"2025-11-12T20:46:36.587806Z"}}
53812025-11-12T20:47:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
53822025-11-12T20:47:35.412ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1050, tv_nsec: 906014195 } })
53832025-11-12T20:47:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
53842025-11-12T20:47:35.412ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
53852025-11-12T20:47:35.412ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
53862025-11-12T20:47:35.412ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
53872025-11-12T20:47:35.412ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 131.371499ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
53882025-11-12T20:47:35.412ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
53892025-11-12T20:47:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
53902025-11-12T20:47:35.413ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53912025-11-12T20:47:35.413ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 132.398687ms iteration = 5 status = 0
53922025-11-12T20:47:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53932025-11-12T20:47:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
53942025-11-12T20:47:35.413ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
53952025-11-12T20:47:35.414ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53962025-11-12T20:47:35.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
53972025-11-12T20:47:35.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53982025-11-12T20:47:35.414ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53992025-11-12T20:47:35.417ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 136.02659ms iteration = 3 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
54002025-11-12T20:47:35.417ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 130.390335ms iteration = 9 status = "NoSitrep"
54012025-11-12T20:47:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54022025-11-12T20:47:35.418ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54032025-11-12T20:47:35.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54042025-11-12T20:47:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54052025-11-12T20:47:35.418ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
54062025-11-12T20:47:35.418ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 137.624741ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54072025-11-12T20:47:35.419ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54082025-11-12T20:47:35.420ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 138.68218ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
54092025-11-12T20:47:35.420ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
54102025-11-12T20:47:35.420ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 139.702596ms iteration = 25 status = {"config_updated":false}
54112025-11-12T20:47:35.422ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
54122025-11-12T20:47:35.422ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54132025-11-12T20:47:35.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54142025-11-12T20:47:35.423ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54152025-11-12T20:47:35.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54162025-11-12T20:47:35.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
54172025-11-12T20:47:35.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54182025-11-12T20:47:35.424ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
54192025-11-12T20:47:35.424ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
54202025-11-12T20:47:35.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54212025-11-12T20:47:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
54222025-11-12T20:47:35.425ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
54232025-11-12T20:47:35.425ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54242025-11-12T20:47:35.426ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54252025-11-12T20:47:35.428ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
54262025-11-12T20:47:35.428ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
54272025-11-12T20:47:35.428ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6a7b38f-2bb5-4818-88c5-1d863dc1de1c", "content-length": "78", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
54282025-11-12T20:47:35.428ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
54292025-11-12T20:47:35.428ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
54302025-11-12T20:47:35.428ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
54312025-11-12T20:47:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
54322025-11-12T20:47:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
54332025-11-12T20:47:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
54342025-11-12T20:47:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
54352025-11-12T20:47:35.430ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
54362025-11-12T20:47:35.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54372025-11-12T20:47:35.430ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54382025-11-12T20:47:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54392025-11-12T20:47:35.430ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
54402025-11-12T20:47:35.430ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 136.683857ms iteration = 9 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
54412025-11-12T20:47:35.433ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54422025-11-12T20:47:35.433ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
54432025-11-12T20:47:35.433ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54442025-11-12T20:47:35.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
54452025-11-12T20:47:35.435ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
54462025-11-12T20:47:35.435ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
54472025-11-12T20:47:35.435ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
54482025-11-12T20:47:35.438ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
54492025-11-12T20:47:35.439ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
54502025-11-12T20:47:35.439ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
54512025-11-12T20:47:35.439ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
54522025-11-12T20:47:35.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
54532025-11-12T20:47:35.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
54542025-11-12T20:47:35.448ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54552025-11-12T20:47:35.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54562025-11-12T20:47:35.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54572025-11-12T20:47:35.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54582025-11-12T20:47:35.456ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
54592025-11-12T20:47:35.460ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
54602025-11-12T20:47:35.464ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
54612025-11-12T20:47:35.467ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
54622025-11-12T20:47:35.467ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 186.66104ms iteration = 13 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
54632025-11-12T20:47:35.468ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
54642025-11-12T20:47:35.469ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
54652025-11-12T20:47:35.469ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
54662025-11-12T20:47:35.469ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
54672025-11-12T20:47:35.469ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54682025-11-12T20:47:35.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54692025-11-12T20:47:35.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54702025-11-12T20:47:35.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54712025-11-12T20:47:35.478ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
54722025-11-12T20:47:35.479ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
54732025-11-12T20:47:35.480ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54742025-11-12T20:47:35.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54752025-11-12T20:47:35.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54762025-11-12T20:47:35.481ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
54772025-11-12T20:47:35.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54782025-11-12T20:47:35.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54792025-11-12T20:47:35.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54802025-11-12T20:47:35.484ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
54812025-11-12T20:47:35.484ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
54822025-11-12T20:47:35.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
54832025-11-12T20:47:35.484ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
54842025-11-12T20:47:35.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
54852025-11-12T20:47:35.485ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = SledList step id = Fetch
54862025-11-12T20:47:35.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
54872025-11-12T20:47:35.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
54882025-11-12T20:47:35.487ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
54892025-11-12T20:47:35.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54902025-11-12T20:47:35.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54912025-11-12T20:47:35.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54922025-11-12T20:47:35.489ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54932025-11-12T20:47:35.490ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
54942025-11-12T20:47:35.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
54952025-11-12T20:47:35.493ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54962025-11-12T20:47:35.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
54972025-11-12T20:47:35.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
54982025-11-12T20:47:35.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
54992025-11-12T20:47:35.495ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55002025-11-12T20:47:35.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55012025-11-12T20:47:35.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55022025-11-12T20:47:35.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 214.927962ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
55032025-11-12T20:47:35.497ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
55042025-11-12T20:47:35.497ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55052025-11-12T20:47:35.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
55062025-11-12T20:47:35.498ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
55072025-11-12T20:47:35.498ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
55082025-11-12T20:47:35.498ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
55092025-11-12T20:47:35.498ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 220.032404ms iteration = 3 status = {"generation":2}
55102025-11-12T20:47:35.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55112025-11-12T20:47:35.499ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
55122025-11-12T20:47:35.499ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
55132025-11-12T20:47:35.499ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 217.761285ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
55142025-11-12T20:47:35.503ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 222.826927ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
55152025-11-12T20:47:35.505ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
55162025-11-12T20:47:35.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55172025-11-12T20:47:35.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55182025-11-12T20:47:35.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55192025-11-12T20:47:35.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55202025-11-12T20:47:35.507ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
55212025-11-12T20:47:35.507ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55222025-11-12T20:47:35.507ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "78fe7c29-057b-4fc1-b0b7-1aa285171d3f", "content-length": "26", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
55232025-11-12T20:47:35.507ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55242025-11-12T20:47:35.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55252025-11-12T20:47:35.508ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = DeployNexusRecords step id = Ensure
55262025-11-12T20:47:35.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55272025-11-12T20:47:35.508ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
55282025-11-12T20:47:35.508ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 227.925428ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
55292025-11-12T20:47:35.509ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55302025-11-12T20:47:35.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55312025-11-12T20:47:35.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55322025-11-12T20:47:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
55332025-11-12T20:47:35.512ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55342025-11-12T20:47:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55352025-11-12T20:47:35.513ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
55362025-11-12T20:47:35.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55372025-11-12T20:47:35.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55382025-11-12T20:47:35.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
55392025-11-12T20:47:35.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
55402025-11-12T20:47:35.515ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
55412025-11-12T20:47:35.515ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 234.105419ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
55422025-11-12T20:47:35.517ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
55432025-11-12T20:47:35.519ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
55442025-11-12T20:47:35.519ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
55452025-11-12T20:47:35.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55462025-11-12T20:47:35.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55472025-11-12T20:47:35.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
55482025-11-12T20:47:35.521ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
55492025-11-12T20:47:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
55502025-11-12T20:47:35.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55512025-11-12T20:47:35.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
55522025-11-12T20:47:35.528ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55532025-11-12T20:47:35.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55542025-11-12T20:47:35.529ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55552025-11-12T20:47:35.529ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
55562025-11-12T20:47:35.529ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55572025-11-12T20:47:35.530ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 248.994104ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55582025-11-12T20:47:35.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
55592025-11-12T20:47:35.532ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
55602025-11-12T20:47:35.533ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
55612025-11-12T20:47:35.533ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
55622025-11-12T20:47:35.533ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 254.922037ms iteration = 3 status = {"generation":1}
55632025-11-12T20:47:35.533ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
55642025-11-12T20:47:35.534ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
55652025-11-12T20:47:35.534ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "495634ff-05ae-4512-95af-6352ae1f9858", "content-length": "81", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
55662025-11-12T20:47:35.541ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
55672025-11-12T20:47:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
55682025-11-12T20:47:35.544ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
55692025-11-12T20:47:35.544ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55702025-11-12T20:47:35.544ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0553395b-f9d7-4295-a902-72f3fa3f4d91", "content-length": "26", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
55712025-11-12T20:47:35.544ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55722025-11-12T20:47:35.546ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
55732025-11-12T20:47:35.555ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
55742025-11-12T20:47:35.555ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
55752025-11-12T20:47:35.556ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "95aafdfc-0221-4d28-8866-75a705ef9eed", "content-length": "81", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
55762025-11-12T20:47:35.562ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
55772025-11-12T20:47:35.566ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
55782025-11-12T20:47:35.568ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 290.227411ms iteration = 5 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
55792025-11-12T20:47:35.575ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55802025-11-12T20:47:35.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
55812025-11-12T20:47:35.578ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
55822025-11-12T20:47:35.578ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 300.152898ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
55832025-11-12T20:47:35.584ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
55842025-11-12T20:47:35.584ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
55852025-11-12T20:47:35.585ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6063e6ac-c5fd-4086-84fe-3829f33e0eb5", "content-length": "81", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
55862025-11-12T20:47:35.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55872025-11-12T20:47:35.587ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 306.41735ms iteration = 5 status = {"disabled":false,"errors":[],"sps":[]}
55882025-11-12T20:47:35.589ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
55892025-11-12T20:47:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
55902025-11-12T20:47:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
55912025-11-12T20:47:35.592ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55922025-11-12T20:47:35.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
55932025-11-12T20:47:35.598ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
55942025-11-12T20:47:35.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
55952025-11-12T20:47:35.613ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55962025-11-12T20:47:35.613ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
55972025-11-12T20:47:35.613ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 332.337431ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
55982025-11-12T20:47:35.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55992025-11-12T20:47:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56002025-11-12T20:47:35.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56012025-11-12T20:47:35.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
56022025-11-12T20:47:35.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56032025-11-12T20:47:35.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56042025-11-12T20:47:35.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56052025-11-12T20:47:35.626ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
56062025-11-12T20:47:35.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56072025-11-12T20:47:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56082025-11-12T20:47:35.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = SledAgent step id = Ensure
56092025-11-12T20:47:35.660ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 378.839724ms iteration = 9 status = {"physical_disks_added":0}
56102025-11-12T20:47:35.660ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56112025-11-12T20:47:35.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56122025-11-12T20:47:35.665ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56132025-11-12T20:47:35.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
56142025-11-12T20:47:35.668ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3144e64b-6825-407c-ae86-c1fbaf9f8d54", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56152025-11-12T20:47:35.668ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56162025-11-12T20:47:35.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56172025-11-12T20:47:35.669ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56182025-11-12T20:47:35.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56192025-11-12T20:47:35.669ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b210f330-149d-4599-b8b6-8a13212cbc39", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56202025-11-12T20:47:35.669ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56212025-11-12T20:47:35.671ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56222025-11-12T20:47:35.671ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
56232025-11-12T20:47:35.671ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 393.566365ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
56242025-11-12T20:47:35.674ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
56252025-11-12T20:47:35.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56262025-11-12T20:47:35.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56272025-11-12T20:47:35.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56282025-11-12T20:47:35.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56292025-11-12T20:47:35.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56302025-11-12T20:47:35.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56312025-11-12T20:47:35.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56322025-11-12T20:47:35.685ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 407.556458ms iteration = 5 status = {}
56332025-11-12T20:47:35.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56342025-11-12T20:47:35.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56352025-11-12T20:47:35.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56362025-11-12T20:47:35.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56372025-11-12T20:47:35.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56382025-11-12T20:47:35.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56392025-11-12T20:47:35.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
56402025-11-12T20:47:35.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56412025-11-12T20:47:35.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56422025-11-12T20:47:35.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56432025-11-12T20:47:35.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56442025-11-12T20:47:35.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56452025-11-12T20:47:35.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56462025-11-12T20:47:35.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56472025-11-12T20:47:35.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56482025-11-12T20:47:35.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56492025-11-12T20:47:35.713ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
56502025-11-12T20:47:35.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56512025-11-12T20:47:35.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56522025-11-12T20:47:35.715ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
56532025-11-12T20:47:35.715ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56542025-11-12T20:47:35.715ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4b35858-a6c3-49ae-9d4a-99a5f3d5e5ef", "content-length": "372", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
56552025-11-12T20:47:35.715ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56562025-11-12T20:47:35.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56572025-11-12T20:47:35.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56582025-11-12T20:47:35.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56592025-11-12T20:47:35.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56602025-11-12T20:47:35.719ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
56612025-11-12T20:47:35.719ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 437.713117ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
56622025-11-12T20:47:35.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
56632025-11-12T20:47:35.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56642025-11-12T20:47:35.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56652025-11-12T20:47:35.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56662025-11-12T20:47:35.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56672025-11-12T20:47:35.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56682025-11-12T20:47:35.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
56692025-11-12T20:47:35.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56702025-11-12T20:47:35.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56712025-11-12T20:47:35.733ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
56722025-11-12T20:47:35.733ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
56732025-11-12T20:47:35.734ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "32198af6-0348-45d1-8ca6-cde3e16e3a0b", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
56742025-11-12T20:47:35.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56752025-11-12T20:47:35.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56762025-11-12T20:47:35.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56772025-11-12T20:47:35.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56782025-11-12T20:47:35.739ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56792025-11-12T20:47:35.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56802025-11-12T20:47:35.742ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
56812025-11-12T20:47:35.742ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
56822025-11-12T20:47:35.743ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 236671141 } })
56832025-11-12T20:47:35.743ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "078709cf-a28b-4f3c-a675-4f5971fb50d9", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
56842025-11-12T20:47:35.744ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 463.218138ms iteration = 7 status = {}
56852025-11-12T20:47:35.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56862025-11-12T20:47:35.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56872025-11-12T20:47:35.754ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
56882025-11-12T20:47:35.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56892025-11-12T20:47:35.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56902025-11-12T20:47:35.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56912025-11-12T20:47:35.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56922025-11-12T20:47:35.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56932025-11-12T20:47:35.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56942025-11-12T20:47:35.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56952025-11-12T20:47:35.765ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56962025-11-12T20:47:35.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56972025-11-12T20:47:35.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56982025-11-12T20:47:35.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56992025-11-12T20:47:35.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57002025-11-12T20:47:35.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57012025-11-12T20:47:35.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57022025-11-12T20:47:35.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57032025-11-12T20:47:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
57042025-11-12T20:47:35.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57052025-11-12T20:47:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57062025-11-12T20:47:35.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57072025-11-12T20:47:35.781ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
57082025-11-12T20:47:35.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57092025-11-12T20:47:35.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57102025-11-12T20:47:35.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57112025-11-12T20:47:35.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57122025-11-12T20:47:35.785ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
57132025-11-12T20:47:35.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57142025-11-12T20:47:35.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57152025-11-12T20:47:35.786ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "63ae8b04-b08b-4d32-bafe-3cfed7d94bfc", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
57162025-11-12T20:47:35.786ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
57172025-11-12T20:47:35.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57182025-11-12T20:47:35.788ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
57192025-11-12T20:47:35.788ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 494.706722ms iteration = 3 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
57202025-11-12T20:47:35.788ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Dependency
57212025-11-12T20:47:35.788ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
57222025-11-12T20:47:35.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57232025-11-12T20:47:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57242025-11-12T20:47:35.790ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57252025-11-12T20:47:35.791ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
57262025-11-12T20:47:35.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57272025-11-12T20:47:35.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57282025-11-12T20:47:35.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57292025-11-12T20:47:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57302025-11-12T20:47:35.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57312025-11-12T20:47:35.793ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57322025-11-12T20:47:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57332025-11-12T20:47:35.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57342025-11-12T20:47:35.795ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57352025-11-12T20:47:35.797ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
57362025-11-12T20:47:35.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57372025-11-12T20:47:35.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57382025-11-12T20:47:35.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
57392025-11-12T20:47:35.815ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
57402025-11-12T20:47:35.815ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "74d4d41b-aa42-4a1b-966c-51da97e239e7", "content-length": "691", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
57412025-11-12T20:47:35.815ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
57422025-11-12T20:47:35.815ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
57432025-11-12T20:47:35.815ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
57442025-11-12T20:47:35.817ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 536.269678ms iteration = 5 status = {}
57452025-11-12T20:47:35.821ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
57462025-11-12T20:47:35.822ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "f81cdcfb-2847-4f40-aa51-a49ecec6ae6d", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
57472025-11-12T20:47:35.824ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = FirewallRules step id = Ensure
57482025-11-12T20:47:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57492025-11-12T20:47:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57502025-11-12T20:47:35.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57512025-11-12T20:47:35.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57522025-11-12T20:47:35.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57532025-11-12T20:47:35.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57542025-11-12T20:47:35.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57552025-11-12T20:47:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57562025-11-12T20:47:35.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57572025-11-12T20:47:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57582025-11-12T20:47:35.841ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
57592025-11-12T20:47:35.842ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "6764d2be-56f3-46b2-82d7-806aab9c9ed5", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57602025-11-12T20:47:35.842ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57612025-11-12T20:47:35.843ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "21a535c8-30fc-48bb-a6ea-5da97d7709cc", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57622025-11-12T20:47:35.844ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57632025-11-12T20:47:35.845ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "96193866-3f63-48be-90ca-beef30de31a9", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57642025-11-12T20:47:35.845ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57652025-11-12T20:47:35.847ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
57662025-11-12T20:47:35.847ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 569.894888ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
57672025-11-12T20:47:35.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57682025-11-12T20:47:35.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57692025-11-12T20:47:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57702025-11-12T20:47:35.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57712025-11-12T20:47:35.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57722025-11-12T20:47:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
57732025-11-12T20:47:35.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57742025-11-12T20:47:35.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57752025-11-12T20:47:35.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57762025-11-12T20:47:35.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57772025-11-12T20:47:35.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57782025-11-12T20:47:35.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57792025-11-12T20:47:35.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57802025-11-12T20:47:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
57812025-11-12T20:47:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57822025-11-12T20:47:35.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57832025-11-12T20:47:35.874ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
57842025-11-12T20:47:35.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57852025-11-12T20:47:35.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57862025-11-12T20:47:35.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57872025-11-12T20:47:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57882025-11-12T20:47:35.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57892025-11-12T20:47:35.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57902025-11-12T20:47:35.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57912025-11-12T20:47:35.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57922025-11-12T20:47:35.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
57932025-11-12T20:47:35.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57942025-11-12T20:47:35.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57952025-11-12T20:47:35.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57962025-11-12T20:47:35.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57972025-11-12T20:47:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57982025-11-12T20:47:35.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
57992025-11-12T20:47:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58002025-11-12T20:47:35.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58012025-11-12T20:47:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58022025-11-12T20:47:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58032025-11-12T20:47:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
58042025-11-12T20:47:35.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
58052025-11-12T20:47:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
58062025-11-12T20:47:35.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58072025-11-12T20:47:35.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58082025-11-12T20:47:35.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58092025-11-12T20:47:35.890ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
58102025-11-12T20:47:35.890ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
58112025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
58122025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
58132025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58142025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58152025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58162025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
58172025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
58182025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
58192025-11-12T20:47:35.891ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
58202025-11-12T20:47:35.891ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58212025-11-12T20:47:35.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58222025-11-12T20:47:35.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
58232025-11-12T20:47:35.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58242025-11-12T20:47:35.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58252025-11-12T20:47:35.895ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
58262025-11-12T20:47:35.895ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
58272025-11-12T20:47:35.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58282025-11-12T20:47:35.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58292025-11-12T20:47:35.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58302025-11-12T20:47:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58312025-11-12T20:47:35.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58322025-11-12T20:47:35.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58332025-11-12T20:47:35.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58342025-11-12T20:47:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58352025-11-12T20:47:35.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58362025-11-12T20:47:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58372025-11-12T20:47:35.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58382025-11-12T20:47:35.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58392025-11-12T20:47:35.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58402025-11-12T20:47:35.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
58412025-11-12T20:47:35.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58422025-11-12T20:47:35.913ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
58432025-11-12T20:47:35.913ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
58442025-11-12T20:47:35.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58452025-11-12T20:47:35.914ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a1bc4f7-55c4-4aaa-957e-45d61c51648f", "content-length": "372", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
58462025-11-12T20:47:35.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58472025-11-12T20:47:35.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58482025-11-12T20:47:35.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58492025-11-12T20:47:35.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58502025-11-12T20:47:35.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58512025-11-12T20:47:35.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58522025-11-12T20:47:35.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58532025-11-12T20:47:35.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58542025-11-12T20:47:35.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58552025-11-12T20:47:35.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58562025-11-12T20:47:35.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58572025-11-12T20:47:35.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58582025-11-12T20:47:35.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58592025-11-12T20:47:35.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58602025-11-12T20:47:35.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58612025-11-12T20:47:35.928ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
58622025-11-12T20:47:35.928ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
58632025-11-12T20:47:35.928ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = cdfec990-77a5-40b6-97b1-0ca2a689a010 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
58642025-11-12T20:47:35.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
58652025-11-12T20:47:35.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
58662025-11-12T20:47:35.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58672025-11-12T20:47:35.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
58682025-11-12T20:47:35.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58692025-11-12T20:47:35.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58702025-11-12T20:47:35.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
58712025-11-12T20:47:35.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
58722025-11-12T20:47:35.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
58732025-11-12T20:47:35.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58742025-11-12T20:47:35.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58752025-11-12T20:47:35.936ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
58762025-11-12T20:47:35.936ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
58772025-11-12T20:47:35.937ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 655.916494ms iteration = 9 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
58782025-11-12T20:47:35.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58792025-11-12T20:47:35.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58802025-11-12T20:47:35.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58812025-11-12T20:47:35.943ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
58822025-11-12T20:47:35.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58832025-11-12T20:47:35.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58842025-11-12T20:47:35.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58852025-11-12T20:47:35.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58862025-11-12T20:47:35.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58872025-11-12T20:47:35.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58882025-11-12T20:47:35.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58892025-11-12T20:47:35.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58902025-11-12T20:47:35.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58912025-11-12T20:47:35.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58922025-11-12T20:47:35.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58932025-11-12T20:47:35.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58942025-11-12T20:47:35.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58952025-11-12T20:47:35.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58962025-11-12T20:47:35.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
58972025-11-12T20:47:35.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
58982025-11-12T20:47:35.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58992025-11-12T20:47:35.963ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
59002025-11-12T20:47:35.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59012025-11-12T20:47:35.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59022025-11-12T20:47:35.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59032025-11-12T20:47:35.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59042025-11-12T20:47:35.965ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
59052025-11-12T20:47:35.965ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
59062025-11-12T20:47:35.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59072025-11-12T20:47:35.966ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9334f07c-b505-42e5-9b82-45da53e741f0", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
59082025-11-12T20:47:35.966ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59092025-11-12T20:47:35.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59102025-11-12T20:47:35.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59112025-11-12T20:47:35.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59122025-11-12T20:47:35.968ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59132025-11-12T20:47:35.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59142025-11-12T20:47:35.971ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59152025-11-12T20:47:35.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59162025-11-12T20:47:35.972ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
59172025-11-12T20:47:35.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59182025-11-12T20:47:35.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59192025-11-12T20:47:35.977ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
59202025-11-12T20:47:35.977ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59212025-11-12T20:47:35.981ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
59222025-11-12T20:47:35.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59232025-11-12T20:47:35.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59242025-11-12T20:47:35.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59252025-11-12T20:47:35.985ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
59262025-11-12T20:47:35.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59272025-11-12T20:47:35.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59282025-11-12T20:47:35.986ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a23f702d-3756-41c4-bc17-983b9c864800", "date": "Wed, 12 Nov 2025 20:47:35 GMT"} })
59292025-11-12T20:47:35.987ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
59302025-11-12T20:47:35.987ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
59312025-11-12T20:47:35.987ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
59322025-11-12T20:47:35.988ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 200.050636ms iteration = 8 status = {}
59332025-11-12T20:47:36.109ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c408b579-148c-4548-80c5-64b93b6e3794", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
59342025-11-12T20:47:36.110ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
59352025-11-12T20:47:36.110ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
59362025-11-12T20:47:36.116ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9212b4dd-f646-4f2c-946c-81cdb37f8708", "content-length": "682", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
59372025-11-12T20:47:36.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59382025-11-12T20:47:36.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59392025-11-12T20:47:36.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59402025-11-12T20:47:36.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59412025-11-12T20:47:36.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59422025-11-12T20:47:36.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59432025-11-12T20:47:36.128ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59442025-11-12T20:47:36.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59452025-11-12T20:47:36.130ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
59462025-11-12T20:47:36.130ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
59472025-11-12T20:47:36.135ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 854.178904ms iteration = 5 status = {}
59482025-11-12T20:47:36.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59492025-11-12T20:47:36.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59502025-11-12T20:47:36.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59512025-11-12T20:47:36.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59522025-11-12T20:47:36.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59532025-11-12T20:47:36.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
59542025-11-12T20:47:36.171ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
59552025-11-12T20:47:36.178ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "69da308a-201b-4164-873c-5f1d359ac2ac", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
59562025-11-12T20:47:36.180ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
59572025-11-12T20:47:36.180ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = Dns step id = Ensure
59582025-11-12T20:47:36.180ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
59592025-11-12T20:47:36.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59602025-11-12T20:47:36.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59612025-11-12T20:47:36.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59622025-11-12T20:47:36.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59632025-11-12T20:47:36.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59642025-11-12T20:47:36.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59652025-11-12T20:47:36.188ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
59662025-11-12T20:47:36.188ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
59672025-11-12T20:47:36.188ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
59682025-11-12T20:47:36.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59692025-11-12T20:47:36.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59702025-11-12T20:47:36.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59712025-11-12T20:47:36.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59722025-11-12T20:47:36.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59732025-11-12T20:47:36.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59742025-11-12T20:47:36.232ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
59752025-11-12T20:47:36.232ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
59762025-11-12T20:47:36.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59772025-11-12T20:47:36.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59782025-11-12T20:47:36.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59792025-11-12T20:47:36.241ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59802025-11-12T20:47:36.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59812025-11-12T20:47:36.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59822025-11-12T20:47:36.244ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
59832025-11-12T20:47:36.244ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
59842025-11-12T20:47:36.244ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
59852025-11-12T20:47:36.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59862025-11-12T20:47:36.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59872025-11-12T20:47:36.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59882025-11-12T20:47:36.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59892025-11-12T20:47:36.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59902025-11-12T20:47:36.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59912025-11-12T20:47:36.291ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
59922025-11-12T20:47:36.291ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
59932025-11-12T20:47:36.291ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
59942025-11-12T20:47:36.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59952025-11-12T20:47:36.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59962025-11-12T20:47:36.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59972025-11-12T20:47:36.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
59982025-11-12T20:47:36.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59992025-11-12T20:47:36.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60002025-11-12T20:47:36.298ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
60012025-11-12T20:47:36.298ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
60022025-11-12T20:47:36.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60032025-11-12T20:47:36.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60042025-11-12T20:47:36.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60052025-11-12T20:47:36.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
60062025-11-12T20:47:36.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60072025-11-12T20:47:36.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60082025-11-12T20:47:36.306ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
60092025-11-12T20:47:36.306ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
60102025-11-12T20:47:36.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60112025-11-12T20:47:36.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60122025-11-12T20:47:36.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60132025-11-12T20:47:36.309ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
60142025-11-12T20:47:36.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60152025-11-12T20:47:36.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60162025-11-12T20:47:36.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60172025-11-12T20:47:36.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60182025-11-12T20:47:36.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60192025-11-12T20:47:36.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60202025-11-12T20:47:36.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60212025-11-12T20:47:36.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
60222025-11-12T20:47:36.318ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
60232025-11-12T20:47:36.318ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
60242025-11-12T20:47:36.318ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = OmicronZones step id = Cleanup
60252025-11-12T20:47:36.318ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = OmicronZones step id = Cleanup
60262025-11-12T20:47:36.318ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = PhysicalDisks step id = Cleanup
60272025-11-12T20:47:36.318ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = Clickhouse step id = Ensure
60282025-11-12T20:47:36.318ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = Clickhouse step id = Ensure
60292025-11-12T20:47:36.349ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
60302025-11-12T20:47:36.354ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "fcb4653e-83cb-46b3-8472-ce8de2f8fc6e", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60312025-11-12T20:47:36.355ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = SupportBundles step id = Cleanup
60322025-11-12T20:47:36.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60332025-11-12T20:47:36.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60342025-11-12T20:47:36.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60352025-11-12T20:47:36.358ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
60362025-11-12T20:47:36.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60372025-11-12T20:47:36.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60382025-11-12T20:47:36.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60392025-11-12T20:47:36.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60402025-11-12T20:47:36.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60412025-11-12T20:47:36.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
60422025-11-12T20:47:36.408ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = OmicronZones step id = Cleanup
60432025-11-12T20:47:36.408ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
60442025-11-12T20:47:36.408ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
60452025-11-12T20:47:36.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60462025-11-12T20:47:36.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60472025-11-12T20:47:36.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60482025-11-12T20:47:36.411ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
60492025-11-12T20:47:36.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60502025-11-12T20:47:36.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60512025-11-12T20:47:36.450ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
60522025-11-12T20:47:36.450ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
60532025-11-12T20:47:36.450ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = Cockroach step id = Ensure
60542025-11-12T20:47:36.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60552025-11-12T20:47:36.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60562025-11-12T20:47:36.466ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
60572025-11-12T20:47:36.466ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b72e8a0-6c34-41a5-a6fe-238175b0dbac step component = MgsUpdates step id = Ensure
60582025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Dependency
60592025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Dependency
60602025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60612025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60622025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
60632025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.188636539s iteration = 4 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3980,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":207212402,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":207212353,"secs":0}},"event_index":1,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":207246713,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":23329274,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":23329224,"secs":0}},"event_index":2,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":230592638,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":144028647,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":144028597,"secs":0}},"event_index":3,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":374639235,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":171242531,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":171242502,"secs":0}},"event_index":4,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":545905017,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":356276479,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":356276429,"secs":0}},"event_index":5,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":902192497,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":138480044,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":138480004,"secs":0}},"event_index":6,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":40686451,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4631,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4591,"secs":0}},"event_index":7,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":40701421,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2120,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2080,"secs":0}},"event_index":8,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":40706361,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3231,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3201,"secs":0}},"event_index":9,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":40711802,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1720,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1690,"secs":0}},"event_index":10,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":40714942,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36258568,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36258528,"secs":0}},"event_index":11,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":76975030,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":53126055,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":53126015,"secs":0}},"event_index":12,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":130110715,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41953345,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41953305,"secs":0}},"event_index":13,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":172081621,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16103397,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16103346,"secs":0}},"event_index":14,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188199849,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4010,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3970,"secs":0}},"event_index":15,"execution_id":"96395048-200f-4dbc-9849-d1ee19e3dbfb","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188208999,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
60642025-11-12T20:47:36.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60652025-11-12T20:47:36.468ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60662025-11-12T20:47:36.468ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60672025-11-12T20:47:36.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60682025-11-12T20:47:36.471ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60692025-11-12T20:47:36.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60702025-11-12T20:47:36.471ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
60712025-11-12T20:47:36.471ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60722025-11-12T20:47:36.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
60732025-11-12T20:47:36.500ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(d0500078-19af-4275-a420-eb4354ebb2e5, 0), (1b79856a-d0c9-4661-821f-cb877e3669c1, 0), (be6f941b-6853-4193-8c1b-4c65d2be0df1, 0)]
60742025-11-12T20:47:36.500ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 994543884 } })
60752025-11-12T20:47:36.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60762025-11-12T20:47:36.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60772025-11-12T20:47:36.511ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60782025-11-12T20:47:36.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60792025-11-12T20:47:36.514ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60802025-11-12T20:47:36.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60812025-11-12T20:47:36.516ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
60822025-11-12T20:47:36.516ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 50.144168ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
60832025-11-12T20:47:36.536ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1052, tv_nsec: 30430552 } })
60842025-11-12T20:47:36.537ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
60852025-11-12T20:47:36.537ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
60862025-11-12T20:47:36.537ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
60872025-11-12T20:47:36.537ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfbb9f70-fbbf-4d4e-8cf9-4d6c9a8b2d56", "content-length": "476", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60882025-11-12T20:47:36.538ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
60892025-11-12T20:47:36.538ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccf08cf3-310f-4f51-a150-cf494ea98d48", "content-length": "734", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60902025-11-12T20:47:36.538ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
60912025-11-12T20:47:36.538ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a24de85-bfec-4cb0-af7b-ee92def6891f", "content-length": "111", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60922025-11-12T20:47:36.538ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
60932025-11-12T20:47:36.539ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "32a5503c-da91-4762-8428-f0428e416cc1", "content-length": "111", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60942025-11-12T20:47:36.539ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
60952025-11-12T20:47:36.539ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b22c25be-4640-4979-87fd-6ee5262c535a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60962025-11-12T20:47:36.539ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
60972025-11-12T20:47:36.540ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a94213d-1bcc-4de8-be4f-c09712134962", "content-length": "163", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
60982025-11-12T20:47:36.540ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
60992025-11-12T20:47:36.540ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dad60b7-1955-4081-90b5-aab7c034a470", "content-length": "166", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61002025-11-12T20:47:36.540ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
61012025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ae70862-457f-419c-8249-411c9d732965", "content-length": "166", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61022025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
61032025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb787c82-13f9-4c32-bcc4-f1331b0989bb", "content-length": "702", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61042025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
61052025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5266fe82-a7cc-487c-bb59-950494b479ed", "content-length": "727", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61062025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
61072025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9ff55ac-b963-4f8f-bcf7-080f4ca74456", "content-length": "729", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61082025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
61092025-11-12T20:47:36.541ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c03b93de-4665-45ea-aed3-1d2586a0b831", "content-length": "728", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61102025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
61112025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "471dc314-3953-4675-a16c-5c6d6925bddd", "content-length": "722", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61122025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
61132025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b3d97cd-0186-4660-aa92-22134d67a5d7", "content-length": "10", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61142025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
61152025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51738c25-9c85-43ee-b30e-fa764edc610e", "content-length": "144", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61162025-11-12T20:47:36.542ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
61172025-11-12T20:47:36.543ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d79310a4-175f-4a98-b648-fab164cac292", "content-length": "147", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61182025-11-12T20:47:36.543ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
61192025-11-12T20:47:36.543ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f0c1fd7-f93c-4a49-8352-094261927a0a", "content-length": "109", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61202025-11-12T20:47:36.543ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
61212025-11-12T20:47:36.544ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c6f9ec2-b22b-4cae-bcec-ed67c6def6ad", "content-length": "109", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61222025-11-12T20:47:36.544ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
61232025-11-12T20:47:36.544ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51308a7c-1f53-4eb3-875f-d1296bd38684", "content-length": "163", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61242025-11-12T20:47:36.544ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
61252025-11-12T20:47:36.544ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7db7a199-da7e-471a-98d1-7dd31cd184b2", "content-length": "163", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61262025-11-12T20:47:36.544ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
61272025-11-12T20:47:36.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da938a0e-4544-472e-976d-b68a8e703241", "content-length": "166", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61282025-11-12T20:47:36.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
61292025-11-12T20:47:36.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6db9409f-2ec8-4120-827b-e0c38e921f9f", "content-length": "166", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61302025-11-12T20:47:36.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
61312025-11-12T20:47:36.545ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70b00e5e-276b-41bb-93ab-0b4db90fe96c", "content-length": "702", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61322025-11-12T20:47:36.545ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
61332025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e1afcaa-4bcf-46d3-bc62-8947523c983f", "content-length": "727", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61342025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
61352025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a205d495-f1ca-4435-b2ed-95dea227f712", "content-length": "729", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61362025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
61372025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8235551-7ca8-453c-bc77-3cd3c113cf92", "content-length": "728", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61382025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61392025-11-12T20:47:36.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61402025-11-12T20:47:36.548ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61412025-11-12T20:47:36.550ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61422025-11-12T20:47:36.550ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61432025-11-12T20:47:36.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61442025-11-12T20:47:36.552ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61452025-11-12T20:47:36.554ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61462025-11-12T20:47:36.554ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61472025-11-12T20:47:36.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61482025-11-12T20:47:36.586ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
61492025-11-12T20:47:36.586ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
61502025-11-12T20:47:36.651ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b03ed8ca-ed72-45c0-a9ac-df1c2439038c", "content-length": "95175", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61512025-11-12T20:47:36.653ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61522025-11-12T20:47:36.653ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61532025-11-12T20:47:36.653ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
61542025-11-12T20:47:36.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
61552025-11-12T20:47:36.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
61562025-11-12T20:47:36.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
61572025-11-12T20:47:36.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
61582025-11-12T20:47:36.653ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
61592025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5a98212-0b64-4f86-b3eb-ab239ed0c13c", "content-length": "64", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61602025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
61612025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ebc27e4-e19b-4df4-8e34-aa4c18009e6f", "content-length": "64", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61622025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "95b1a2bc-104e-48ff-a47d-9e54c55459bd", "content-length": "64", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61632025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9fdc30a-33db-4517-8e85-39476a0a3358", "content-length": "64", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61642025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
61652025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
61662025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
61672025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31c6a519-f5de-4028-9e47-77c89ff3021e", "content-length": "64", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61682025-11-12T20:47:36.654ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
61692025-11-12T20:47:36.695ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "74e37851-8fbc-4bbd-b405-59ca954a9778", "content-length": "381009", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61702025-11-12T20:47:36.696ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eef4d41-49ee-4d06-b89d-7b84924e091d", "content-length": "386467", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61712025-11-12T20:47:36.697ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0dc9e6d-5044-4d48-877d-0f0b3321e286", "content-length": "415803", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61722025-11-12T20:47:36.697ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "169d5e3a-3b24-4b1b-8ac3-63cbcacce834", "content-length": "378383", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61732025-11-12T20:47:36.703ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b70dc491-47a6-4b4e-a8c7-a6d99235ab8d", "content-length": "381665", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61742025-11-12T20:47:36.706ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
61752025-11-12T20:47:36.706ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
61762025-11-12T20:47:36.706ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
61772025-11-12T20:47:36.706ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
61782025-11-12T20:47:36.706ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
61792025-11-12T20:47:36.735ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
61802025-11-12T20:47:36.736ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
61812025-11-12T20:47:36.739ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "b127dac8-1854-4605-82b5-4e0cea942fbb", "content-length": "128", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61822025-11-12T20:47:36.739ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
61832025-11-12T20:47:36.830ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
61842025-11-12T20:47:36.831ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
61852025-11-12T20:47:36.831ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "740e8afb-6aae-4ca8-9a84-c041702a480e", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61862025-11-12T20:47:36.831ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
61872025-11-12T20:47:36.833ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
61882025-11-12T20:47:36.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
61892025-11-12T20:47:36.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0aa0058-dc5d-420c-b94e-6419b2b48949", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61902025-11-12T20:47:36.833ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
61912025-11-12T20:47:36.835ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
61922025-11-12T20:47:36.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
61932025-11-12T20:47:36.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "af959b32-d0c2-486a-9adf-94a487de6c72", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:47:36 GMT"} })
61942025-11-12T20:47:36.835ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
61952025-11-12T20:47:36.836ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
61962025-11-12T20:47:36.836ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
61972025-11-12T20:47:36.838ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61982025-11-12T20:47:36.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
61992025-11-12T20:47:36.840ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62002025-11-12T20:47:36.842ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
62012025-11-12T20:47:36.842ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62022025-11-12T20:47:36.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
62032025-11-12T20:47:37.045ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 677c0ea6-1049-4589-9f51-940941493fc6 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
62042025-11-12T20:47:37.045ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 677c0ea6-1049-4589-9f51-940941493fc6 time_started = 2025-11-12 20:47:36.537485 UTC
62052025-11-12T20:47:37.045ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 13 reason = Dependency
62062025-11-12T20:47:37.045ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62072025-11-12T20:47:37.045ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 579.123447ms iteration = 4 status = {"collection_id":"677c0ea6-1049-4589-9f51-940941493fc6","time_done":"2025-11-12 20:47:36.836411 UTC","time_started":"2025-11-12 20:47:36.537485 UTC"}
62082025-11-12T20:47:37.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62092025-11-12T20:47:37.047ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62102025-11-12T20:47:37.050ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
62112025-11-12T20:47:37.050ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62122025-11-12T20:47:37.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62132025-11-12T20:47:37.109ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62142025-11-12T20:47:37.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62152025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
62162025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 10 reason = Dependency
62172025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 118.206409ms iteration = 13 status = {"Loaded":{"collection_id":"4856956c-8555-4ed1-a4d2-383b97af5a6c","time_loaded":"2025-11-12T20:47:37.045534911Z","time_started":"2025-11-12T20:47:36.587908Z"}}
62182025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Dependency
62192025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62202025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62212025-11-12T20:47:37.163ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62222025-11-12T20:47:37.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62232025-11-12T20:47:37.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62242025-11-12T20:47:37.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62252025-11-12T20:47:37.165ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62262025-11-12T20:47:37.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62272025-11-12T20:47:37.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62282025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62292025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62302025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62312025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62322025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62332025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62342025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62352025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62362025-11-12T20:47:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62372025-11-12T20:47:37.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62382025-11-12T20:47:37.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62392025-11-12T20:47:37.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62402025-11-12T20:47:37.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62412025-11-12T20:47:37.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62422025-11-12T20:47:37.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62432025-11-12T20:47:37.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62442025-11-12T20:47:37.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62452025-11-12T20:47:37.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62462025-11-12T20:47:37.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62472025-11-12T20:47:37.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62482025-11-12T20:47:37.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62492025-11-12T20:47:37.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62502025-11-12T20:47:37.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62512025-11-12T20:47:37.176ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62522025-11-12T20:47:37.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62532025-11-12T20:47:37.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62542025-11-12T20:47:37.179ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62552025-11-12T20:47:37.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62562025-11-12T20:47:37.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62572025-11-12T20:47:37.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62582025-11-12T20:47:37.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62592025-11-12T20:47:37.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62602025-11-12T20:47:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62612025-11-12T20:47:37.188ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
62622025-11-12T20:47:37.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62632025-11-12T20:47:37.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62642025-11-12T20:47:37.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62652025-11-12T20:47:37.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62662025-11-12T20:47:37.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62672025-11-12T20:47:37.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62682025-11-12T20:47:37.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62692025-11-12T20:47:37.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
62702025-11-12T20:47:37.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
62712025-11-12T20:47:37.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
62722025-11-12T20:47:37.198ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
62732025-11-12T20:47:37.198ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 35.041067ms iteration = 6 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"4856956c-8555-4ed1-a4d2-383b97af5a6c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
62742025-11-12T20:47:37.221ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 57.60508ms iteration = 10 status = {"physical_disks_added":0}
62752025-11-12T20:47:37.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62762025-11-12T20:47:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62772025-11-12T20:47:37.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62782025-11-12T20:47:37.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62792025-11-12T20:47:37.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62802025-11-12T20:47:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62812025-11-12T20:47:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62822025-11-12T20:47:37.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62832025-11-12T20:47:37.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62842025-11-12T20:47:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62852025-11-12T20:47:37.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62862025-11-12T20:47:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62872025-11-12T20:47:37.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62882025-11-12T20:47:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62892025-11-12T20:47:37.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62902025-11-12T20:47:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62912025-11-12T20:47:37.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62922025-11-12T20:47:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62932025-11-12T20:47:37.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62942025-11-12T20:47:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62952025-11-12T20:47:37.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62962025-11-12T20:47:37.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
62972025-11-12T20:47:37.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62982025-11-12T20:47:37.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62992025-11-12T20:47:37.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63002025-11-12T20:47:37.283ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63012025-11-12T20:47:37.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63022025-11-12T20:47:37.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63032025-11-12T20:47:37.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63042025-11-12T20:47:37.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63052025-11-12T20:47:37.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63062025-11-12T20:47:37.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
63072025-11-12T20:47:37.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63082025-11-12T20:47:37.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63092025-11-12T20:47:37.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63102025-11-12T20:47:37.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63112025-11-12T20:47:37.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63122025-11-12T20:47:37.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
63132025-11-12T20:47:37.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63142025-11-12T20:47:37.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63152025-11-12T20:47:37.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63162025-11-12T20:47:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63172025-11-12T20:47:37.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63182025-11-12T20:47:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
63192025-11-12T20:47:37.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63202025-11-12T20:47:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63212025-11-12T20:47:37.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63222025-11-12T20:47:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63232025-11-12T20:47:37.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63242025-11-12T20:47:37.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63252025-11-12T20:47:37.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63262025-11-12T20:47:37.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63272025-11-12T20:47:37.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63282025-11-12T20:47:37.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63292025-11-12T20:47:37.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63302025-11-12T20:47:37.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63312025-11-12T20:47:37.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63322025-11-12T20:47:37.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63332025-11-12T20:47:37.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63342025-11-12T20:47:37.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63352025-11-12T20:47:37.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63362025-11-12T20:47:37.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63372025-11-12T20:47:37.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63382025-11-12T20:47:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63392025-11-12T20:47:37.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63402025-11-12T20:47:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63412025-11-12T20:47:37.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63422025-11-12T20:47:37.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63432025-11-12T20:47:37.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63442025-11-12T20:47:37.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63452025-11-12T20:47:37.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63462025-11-12T20:47:37.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63472025-11-12T20:47:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63482025-11-12T20:47:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63492025-11-12T20:47:37.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63502025-11-12T20:47:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
63512025-11-12T20:47:37.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63522025-11-12T20:47:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63532025-11-12T20:47:37.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63542025-11-12T20:47:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63552025-11-12T20:47:37.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63562025-11-12T20:47:37.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
63572025-11-12T20:47:37.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63582025-11-12T20:47:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63592025-11-12T20:47:37.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63602025-11-12T20:47:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63612025-11-12T20:47:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63622025-11-12T20:47:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
63632025-11-12T20:47:37.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63642025-11-12T20:47:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63652025-11-12T20:47:37.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63662025-11-12T20:47:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63672025-11-12T20:47:37.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63682025-11-12T20:47:37.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
63692025-11-12T20:47:37.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63702025-11-12T20:47:37.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63712025-11-12T20:47:37.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63722025-11-12T20:47:37.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63732025-11-12T20:47:37.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63742025-11-12T20:47:37.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
63752025-11-12T20:47:37.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63762025-11-12T20:47:37.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63772025-11-12T20:47:37.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63782025-11-12T20:47:37.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63792025-11-12T20:47:37.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63802025-11-12T20:47:37.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
63812025-11-12T20:47:37.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63822025-11-12T20:47:37.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63832025-11-12T20:47:37.391ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
63842025-11-12T20:47:37.391ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
63852025-11-12T20:47:37.391ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9b078800-10d8-470d-9078-4e301761c68d parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
63862025-11-12T20:47:37.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63872025-11-12T20:47:37.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63882025-11-12T20:47:37.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63892025-11-12T20:47:37.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
63902025-11-12T20:47:37.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63912025-11-12T20:47:37.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63922025-11-12T20:47:37.399ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
63932025-11-12T20:47:37.399ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
63942025-11-12T20:47:37.399ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 235.419883ms iteration = 10 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
63952025-11-12T20:47:37.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 307 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 44ef00f2-b1fb-4c99-9367-dffcbbdca002 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
63962025-11-12T20:47:40.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 26 reason = Timeout
63972025-11-12T20:47:40.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
63982025-11-12T20:47:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
63992025-11-12T20:47:40.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64002025-11-12T20:47:40.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64012025-11-12T20:47:40.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64022025-11-12T20:47:40.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64032025-11-12T20:47:40.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
64042025-11-12T20:47:40.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.220247ms iteration = 26 status = {"config_updated":false}
64052025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
64062025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
64072025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64082025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64092025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 27 reason = Timeout
64102025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64112025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64122025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64132025-11-12T20:47:45.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64142025-11-12T20:47:45.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64152025-11-12T20:47:45.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64162025-11-12T20:47:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
64172025-11-12T20:47:45.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64182025-11-12T20:47:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64192025-11-12T20:47:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64202025-11-12T20:47:45.287ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64212025-11-12T20:47:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64222025-11-12T20:47:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64232025-11-12T20:47:45.289ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
64242025-11-12T20:47:45.289ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.964916ms iteration = 27 status = {"config_updated":false}
64252025-11-12T20:47:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64262025-11-12T20:47:45.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64272025-11-12T20:47:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
64282025-11-12T20:47:45.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64292025-11-12T20:47:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64302025-11-12T20:47:45.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
64312025-11-12T20:47:45.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.240593ms iteration = 14 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
64322025-11-12T20:47:45.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64332025-11-12T20:47:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64342025-11-12T20:47:45.352ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.257324ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64352025-11-12T20:47:47.941ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 364 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 125dc9a4-931b-46e9-a15e-5b97534e5eec response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
64362025-11-12T20:47:50.280ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 14 reason = Timeout
64372025-11-12T20:47:50.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
64382025-11-12T20:47:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
64392025-11-12T20:47:50.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 28 reason = Timeout
64402025-11-12T20:47:50.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64412025-11-12T20:47:50.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64422025-11-12T20:47:50.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64432025-11-12T20:47:50.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64442025-11-12T20:47:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
64452025-11-12T20:47:50.286ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
64462025-11-12T20:47:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
64472025-11-12T20:47:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64482025-11-12T20:47:50.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64492025-11-12T20:47:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64502025-11-12T20:47:50.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 10 reason = Timeout
64512025-11-12T20:47:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
64522025-11-12T20:47:50.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
64532025-11-12T20:47:50.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
64542025-11-12T20:47:50.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.372211ms iteration = 28 status = {"config_updated":false}
64552025-11-12T20:47:50.288ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4856956c-8555-4ed1-a4d2-383b97af5a6c original_time_started = 2025-11-12 20:47:36.587908 UTC
64562025-11-12T20:47:50.288ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.014558ms iteration = 14 status = {"Loaded":{"collection_id":"4856956c-8555-4ed1-a4d2-383b97af5a6c","time_loaded":"2025-11-12T20:47:50.280424625Z","time_started":"2025-11-12T20:47:36.587908Z"}}
64572025-11-12T20:47:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64582025-11-12T20:47:50.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
64592025-11-12T20:47:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
64602025-11-12T20:47:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
64612025-11-12T20:47:50.293ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.945864ms iteration = 10 status = "NoSitrep"
64622025-11-12T20:47:50.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 10 reason = Timeout
64632025-11-12T20:47:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64642025-11-12T20:47:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64652025-11-12T20:47:50.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64662025-11-12T20:47:50.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64672025-11-12T20:47:50.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64682025-11-12T20:47:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
64692025-11-12T20:47:50.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64702025-11-12T20:47:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64712025-11-12T20:47:50.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64722025-11-12T20:47:50.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64732025-11-12T20:47:50.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64742025-11-12T20:47:50.303ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
64752025-11-12T20:47:50.304ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
64762025-11-12T20:47:50.304ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.765189ms iteration = 10 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
64772025-11-12T20:47:55.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
64782025-11-12T20:47:55.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
64792025-11-12T20:47:55.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64802025-11-12T20:47:55.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64812025-11-12T20:47:55.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 29 reason = Timeout
64822025-11-12T20:47:55.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64832025-11-12T20:47:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64842025-11-12T20:47:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64852025-11-12T20:47:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64862025-11-12T20:47:55.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64872025-11-12T20:47:55.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64882025-11-12T20:47:55.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64892025-11-12T20:47:55.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64902025-11-12T20:47:55.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.916955ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64912025-11-12T20:47:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
64922025-11-12T20:47:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64932025-11-12T20:47:55.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64942025-11-12T20:47:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64952025-11-12T20:47:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64962025-11-12T20:47:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64972025-11-12T20:47:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64982025-11-12T20:47:55.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64992025-11-12T20:47:55.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
65002025-11-12T20:47:55.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.079224ms iteration = 29 status = {"config_updated":false}
65012025-11-12T20:47:55.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65022025-11-12T20:47:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
65032025-11-12T20:47:55.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65042025-11-12T20:47:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
65052025-11-12T20:47:55.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
65062025-11-12T20:47:55.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.697433ms iteration = 15 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
65072025-11-12T20:47:57.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 297 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = af8e3dbf-460c-405d-b068-249acae9a837 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
65082025-11-12T20:48:00.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 30 reason = Timeout
65092025-11-12T20:48:00.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
65102025-11-12T20:48:00.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
65112025-11-12T20:48:00.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65122025-11-12T20:48:00.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
65132025-11-12T20:48:00.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
65142025-11-12T20:48:00.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
65152025-11-12T20:48:00.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
65162025-11-12T20:48:00.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.898979ms iteration = 30 status = {"config_updated":false}
65172025-11-12T20:48:05.277ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
65182025-11-12T20:48:05.278ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 6 reason = Timeout
65192025-11-12T20:48:05.278ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
65202025-11-12T20:48:05.278ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
65212025-11-12T20:48:05.278ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
65222025-11-12T20:48:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
65232025-11-12T20:48:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
65242025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 15 reason = Timeout
65252025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
65262025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
65272025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
65282025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
65292025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 6 reason = Timeout
65302025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
65312025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
65322025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65332025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65342025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
65352025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65362025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 31 reason = Timeout
65372025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
65382025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
65392025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
65402025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
65412025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65422025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
65432025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
65442025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
65452025-11-12T20:48:05.280ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
65462025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65472025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
65482025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65492025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
65502025-11-12T20:48:05.280ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
65512025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65522025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
65532025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
65542025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65552025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65562025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
65572025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
65582025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
65592025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
65602025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
65612025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
65622025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65632025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
65642025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65652025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
65662025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
65672025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
65682025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
65692025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65702025-11-12T20:48:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1080, tv_nsec: 775313579 } })
65712025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
65722025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65732025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65742025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
65752025-11-12T20:48:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1080, tv_nsec: 775396751 } })
65762025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65772025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
65782025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
65792025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65802025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
65812025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
65822025-11-12T20:48:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65832025-11-12T20:48:05.282ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1080, tv_nsec: 776279204 } })
65842025-11-12T20:48:05.283ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
65852025-11-12T20:48:05.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65862025-11-12T20:48:05.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65872025-11-12T20:48:05.283ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65882025-11-12T20:48:05.284ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65892025-11-12T20:48:05.284ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65902025-11-12T20:48:05.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65912025-11-12T20:48:05.284ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65922025-11-12T20:48:05.284ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.690464ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
65932025-11-12T20:48:05.285ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65942025-11-12T20:48:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65952025-11-12T20:48:05.286ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
65962025-11-12T20:48:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
65972025-11-12T20:48:05.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 11 reason = Timeout
65982025-11-12T20:48:05.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
65992025-11-12T20:48:05.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66002025-11-12T20:48:05.288ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
66012025-11-12T20:48:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
66022025-11-12T20:48:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
66032025-11-12T20:48:05.290ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.801043ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
66042025-11-12T20:48:05.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66052025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
66062025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
66072025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66082025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66092025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
66102025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
66112025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
66122025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
66132025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66142025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
66152025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
66162025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66172025-11-12T20:48:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
66182025-11-12T20:48:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66192025-11-12T20:48:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
66202025-11-12T20:48:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
66212025-11-12T20:48:05.292ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
66222025-11-12T20:48:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
66232025-11-12T20:48:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66242025-11-12T20:48:05.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66252025-11-12T20:48:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
66262025-11-12T20:48:05.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 11 reason = Timeout
66272025-11-12T20:48:05.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
66282025-11-12T20:48:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
66292025-11-12T20:48:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
66302025-11-12T20:48:05.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66312025-11-12T20:48:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
66322025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66332025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
66342025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.412863ms iteration = 31 status = {"config_updated":false}
66352025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66362025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
66372025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
66382025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
66392025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
66402025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
66412025-11-12T20:48:05.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
66422025-11-12T20:48:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
66432025-11-12T20:48:05.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66442025-11-12T20:48:05.297ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66452025-11-12T20:48:05.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66462025-11-12T20:48:05.298ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.755869ms iteration = 11 status = "NoSitrep"
66472025-11-12T20:48:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 18.616042ms iteration = 6 status = 0
66482025-11-12T20:48:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
66492025-11-12T20:48:05.301ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66502025-11-12T20:48:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66512025-11-12T20:48:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
66522025-11-12T20:48:05.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
66532025-11-12T20:48:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
66542025-11-12T20:48:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
66552025-11-12T20:48:05.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66562025-11-12T20:48:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
66572025-11-12T20:48:05.304ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
66582025-11-12T20:48:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.392705ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
66592025-11-12T20:48:05.306ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4856956c-8555-4ed1-a4d2-383b97af5a6c original_time_started = 2025-11-12 20:47:36.587908 UTC
66602025-11-12T20:48:05.306ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 26.443874ms iteration = 15 status = {"Loaded":{"collection_id":"4856956c-8555-4ed1-a4d2-383b97af5a6c","time_loaded":"2025-11-12T20:48:05.280092907Z","time_started":"2025-11-12T20:47:36.587908Z"}}
66612025-11-12T20:48:05.308ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
66622025-11-12T20:48:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
66632025-11-12T20:48:05.309ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
66642025-11-12T20:48:05.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
66652025-11-12T20:48:05.310ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1080, tv_nsec: 803992862 } })
66662025-11-12T20:48:05.310ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
66672025-11-12T20:48:05.311ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66682025-11-12T20:48:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
66692025-11-12T20:48:05.314ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
66702025-11-12T20:48:05.314ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 21.143117ms iteration = 11 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
66712025-11-12T20:48:05.325ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 44.506892ms iteration = 11 status = {"physical_disks_added":0}
66722025-11-12T20:48:05.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
66732025-11-12T20:48:05.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.441501ms iteration = 16 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
66742025-11-12T20:48:05.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66752025-11-12T20:48:05.345ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
66762025-11-12T20:48:05.345ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66772025-11-12T20:48:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66782025-11-12T20:48:05.348ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66792025-11-12T20:48:05.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 68.472442ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
66802025-11-12T20:48:05.351ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
66812025-11-12T20:48:05.351ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66822025-11-12T20:48:05.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66832025-11-12T20:48:05.356ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66842025-11-12T20:48:05.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
66852025-11-12T20:48:05.358ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
66862025-11-12T20:48:05.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
66872025-11-12T20:48:05.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
66882025-11-12T20:48:05.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66892025-11-12T20:48:05.359ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
66902025-11-12T20:48:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
66912025-11-12T20:48:05.361ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66922025-11-12T20:48:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
66932025-11-12T20:48:05.365ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
66942025-11-12T20:48:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
66952025-11-12T20:48:05.367ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
66962025-11-12T20:48:05.367ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 86.719754ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
66972025-11-12T20:48:05.367ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 86.781336ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
66982025-11-12T20:48:05.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 89.594688ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
66992025-11-12T20:48:05.371ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
67002025-11-12T20:48:05.371ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
67012025-11-12T20:48:05.371ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67022025-11-12T20:48:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
67032025-11-12T20:48:05.377ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 96.98977ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
67042025-11-12T20:48:05.391ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
67052025-11-12T20:48:05.391ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "85fdaacf-a063-44fa-9b7d-594b7ae9246f", "content-length": "78", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67062025-11-12T20:48:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
67072025-11-12T20:48:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
67082025-11-12T20:48:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
67092025-11-12T20:48:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
67102025-11-12T20:48:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
67112025-11-12T20:48:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
67122025-11-12T20:48:05.393ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
67132025-11-12T20:48:05.394ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
67142025-11-12T20:48:05.394ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
67152025-11-12T20:48:05.396ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
67162025-11-12T20:48:05.396ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
67172025-11-12T20:48:05.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
67182025-11-12T20:48:05.397ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
67192025-11-12T20:48:05.397ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
67202025-11-12T20:48:05.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
67212025-11-12T20:48:05.397ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
67222025-11-12T20:48:05.397ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
67232025-11-12T20:48:05.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
67242025-11-12T20:48:05.398ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
67252025-11-12T20:48:05.398ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
67262025-11-12T20:48:05.398ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "92c9e42f-12f4-4ee6-9e93-362fb45e310b", "content-length": "81", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67272025-11-12T20:48:05.400ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 119.675217ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
67282025-11-12T20:48:05.413ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
67292025-11-12T20:48:05.414ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 136.940054ms iteration = 6 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
67302025-11-12T20:48:05.416ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
67312025-11-12T20:48:05.416ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
67322025-11-12T20:48:05.416ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
67332025-11-12T20:48:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
67342025-11-12T20:48:05.417ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 136.978245ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
67352025-11-12T20:48:05.419ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
67362025-11-12T20:48:05.419ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
67372025-11-12T20:48:05.420ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "de79d16e-3d7e-4a5d-a59b-56a46ec4e7a8", "content-length": "26", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67382025-11-12T20:48:05.420ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
67392025-11-12T20:48:05.420ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
67402025-11-12T20:48:05.420ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
67412025-11-12T20:48:05.421ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0961c2e3-469b-4ea0-af73-aef3f677fb71", "content-length": "26", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67422025-11-12T20:48:05.421ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
67432025-11-12T20:48:05.422ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
67442025-11-12T20:48:05.423ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
67452025-11-12T20:48:05.443ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
67462025-11-12T20:48:05.443ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
67472025-11-12T20:48:05.444ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
67482025-11-12T20:48:05.444ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
67492025-11-12T20:48:05.444ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "413903b1-40ea-430f-8685-4a5a575c44c7", "content-length": "81", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67502025-11-12T20:48:05.444ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "03deab45-b884-4161-88b1-b7e9ef55d4ae", "content-length": "81", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67512025-11-12T20:48:05.446ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 165.615736ms iteration = 6 status = {"disabled":false,"errors":[],"sps":[]}
67522025-11-12T20:48:05.453ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
67532025-11-12T20:48:05.453ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67542025-11-12T20:48:05.455ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "78a74884-a3dc-42b6-b108-890c92d60490", "content-length": "372", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67552025-11-12T20:48:05.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67562025-11-12T20:48:05.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67572025-11-12T20:48:05.459ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67582025-11-12T20:48:05.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
67592025-11-12T20:48:05.461ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67602025-11-12T20:48:05.463ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
67612025-11-12T20:48:05.463ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67622025-11-12T20:48:05.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
67632025-11-12T20:48:05.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67642025-11-12T20:48:05.467ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
67652025-11-12T20:48:05.467ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67662025-11-12T20:48:05.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
67672025-11-12T20:48:05.470ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
67682025-11-12T20:48:05.470ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
67692025-11-12T20:48:05.470ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 189.843693ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
67702025-11-12T20:48:05.507ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 229.52992ms iteration = 6 status = {}
67712025-11-12T20:48:05.513ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
67722025-11-12T20:48:05.513ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
67732025-11-12T20:48:05.513ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 7674542 } })
67742025-11-12T20:48:05.549ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
67752025-11-12T20:48:05.549ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "76a4049a-9df6-4e92-8b54-ab1788ef4a3e", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67762025-11-12T20:48:05.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67772025-11-12T20:48:05.550ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "068bd5f6-e25c-4eec-919d-443eff93839c", "content-length": "691", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67782025-11-12T20:48:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67792025-11-12T20:48:05.550ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
67802025-11-12T20:48:05.550ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
67812025-11-12T20:48:05.550ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
67822025-11-12T20:48:05.552ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 271.546647ms iteration = 6 status = {}
67832025-11-12T20:48:05.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67842025-11-12T20:48:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67852025-11-12T20:48:05.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67862025-11-12T20:48:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67872025-11-12T20:48:05.557ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
67882025-11-12T20:48:05.557ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67892025-11-12T20:48:05.558ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6743f861-e7ba-4faf-9db3-94120c40e761", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
67902025-11-12T20:48:05.559ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 278.658451ms iteration = 9 status = {}
67912025-11-12T20:48:05.594ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67922025-11-12T20:48:05.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67932025-11-12T20:48:05.596ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
67942025-11-12T20:48:05.596ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67952025-11-12T20:48:05.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67962025-11-12T20:48:05.608ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
67972025-11-12T20:48:05.608ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
67982025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
67992025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
68002025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68012025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68022025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68032025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68042025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68052025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68062025-11-12T20:48:05.609ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
68072025-11-12T20:48:05.609ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68082025-11-12T20:48:05.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68092025-11-12T20:48:05.612ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
68102025-11-12T20:48:05.612ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
68112025-11-12T20:48:05.712ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "fc38ee10-c512-4088-a78d-2800f3a54464", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
68122025-11-12T20:48:05.712ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
68132025-11-12T20:48:05.712ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
68142025-11-12T20:48:05.713ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4839c5a-daea-4717-aa74-ac0b25d0a43c", "content-length": "682", "date": "Wed, 12 Nov 2025 20:48:05 GMT"} })
68152025-11-12T20:48:05.713ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68162025-11-12T20:48:05.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68172025-11-12T20:48:05.717ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68182025-11-12T20:48:05.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68192025-11-12T20:48:05.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68202025-11-12T20:48:05.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68212025-11-12T20:48:05.722ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68222025-11-12T20:48:05.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68232025-11-12T20:48:05.729ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 448.168368ms iteration = 6 status = {}
68242025-11-12T20:48:07.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 306 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 28eb8fb7-ba53-4af9-b586-5c95828e2317 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
68252025-11-12T20:48:10.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 32 reason = Timeout
68262025-11-12T20:48:10.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68272025-11-12T20:48:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68282025-11-12T20:48:10.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68292025-11-12T20:48:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
68302025-11-12T20:48:10.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68312025-11-12T20:48:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68322025-11-12T20:48:10.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
68332025-11-12T20:48:10.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.788031ms iteration = 32 status = {"config_updated":false}
68342025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
68352025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
68362025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68372025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 33 reason = Timeout
68382025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68392025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68402025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68412025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68422025-11-12T20:48:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
68432025-11-12T20:48:15.283ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68442025-11-12T20:48:15.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68452025-11-12T20:48:15.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68462025-11-12T20:48:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68472025-11-12T20:48:15.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.832189ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
68482025-11-12T20:48:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
68492025-11-12T20:48:15.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68502025-11-12T20:48:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68512025-11-12T20:48:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
68522025-11-12T20:48:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68532025-11-12T20:48:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
68542025-11-12T20:48:15.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68552025-11-12T20:48:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
68562025-11-12T20:48:15.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
68572025-11-12T20:48:15.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.979371ms iteration = 33 status = {"config_updated":false}
68582025-11-12T20:48:15.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68592025-11-12T20:48:15.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
68602025-11-12T20:48:15.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68612025-11-12T20:48:15.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
68622025-11-12T20:48:15.312ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
68632025-11-12T20:48:15.312ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.2717ms iteration = 17 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
68642025-11-12T20:48:17.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = be60f39f-093c-4898-8797-f01d0d8b538c response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
68652025-11-12T20:48:20.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 16 reason = Timeout
68662025-11-12T20:48:20.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
68672025-11-12T20:48:20.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
68682025-11-12T20:48:20.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 34 reason = Timeout
68692025-11-12T20:48:20.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68702025-11-12T20:48:20.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68712025-11-12T20:48:20.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68722025-11-12T20:48:20.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68732025-11-12T20:48:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
68742025-11-12T20:48:20.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
68752025-11-12T20:48:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
68762025-11-12T20:48:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
68772025-11-12T20:48:20.286ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
68782025-11-12T20:48:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
68792025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
68802025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.776516ms iteration = 34 status = {"config_updated":false}
68812025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 12 reason = Timeout
68822025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
68832025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4856956c-8555-4ed1-a4d2-383b97af5a6c original_time_started = 2025-11-12 20:47:36.587908 UTC
68842025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.662529ms iteration = 16 status = {"Loaded":{"collection_id":"4856956c-8555-4ed1-a4d2-383b97af5a6c","time_loaded":"2025-11-12T20:48:20.279747751Z","time_started":"2025-11-12T20:47:36.587908Z"}}
68852025-11-12T20:48:20.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
68862025-11-12T20:48:20.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68872025-11-12T20:48:20.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
68882025-11-12T20:48:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
68892025-11-12T20:48:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
68902025-11-12T20:48:20.292ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.280057ms iteration = 12 status = "NoSitrep"
68912025-11-12T20:48:20.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 12 reason = Timeout
68922025-11-12T20:48:20.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
68932025-11-12T20:48:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
68942025-11-12T20:48:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
68952025-11-12T20:48:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
68962025-11-12T20:48:20.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68972025-11-12T20:48:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
68982025-11-12T20:48:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
68992025-11-12T20:48:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
69002025-11-12T20:48:20.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
69012025-11-12T20:48:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
69022025-11-12T20:48:20.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69032025-11-12T20:48:20.303ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
69042025-11-12T20:48:20.304ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
69052025-11-12T20:48:20.304ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.270536ms iteration = 12 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
69062025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
69072025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69082025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
69092025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69102025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 35 reason = Timeout
69112025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69122025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69132025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69142025-11-12T20:48:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69152025-11-12T20:48:25.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69162025-11-12T20:48:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69172025-11-12T20:48:25.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69182025-11-12T20:48:25.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69192025-11-12T20:48:25.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.842289ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69202025-11-12T20:48:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
69212025-11-12T20:48:25.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69222025-11-12T20:48:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69232025-11-12T20:48:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69242025-11-12T20:48:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69252025-11-12T20:48:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69262025-11-12T20:48:25.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
69272025-11-12T20:48:25.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.302895ms iteration = 35 status = {"config_updated":false}
69282025-11-12T20:48:25.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69292025-11-12T20:48:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69302025-11-12T20:48:25.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69312025-11-12T20:48:25.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
69322025-11-12T20:48:25.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69332025-11-12T20:48:25.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69342025-11-12T20:48:25.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
69352025-11-12T20:48:25.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.193006ms iteration = 18 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
69362025-11-12T20:48:27.941ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 377 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 19328fce-ee78-4d1e-9c08-8467ab7c169e response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
69372025-11-12T20:48:30.281ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 36 reason = Timeout
69382025-11-12T20:48:30.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69392025-11-12T20:48:30.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69402025-11-12T20:48:30.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69412025-11-12T20:48:30.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
69422025-11-12T20:48:30.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69432025-11-12T20:48:30.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69442025-11-12T20:48:30.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
69452025-11-12T20:48:30.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.144645ms iteration = 36 status = {"config_updated":false}
69462025-11-12T20:48:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
69472025-11-12T20:48:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
69482025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
69492025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
69502025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
69512025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69522025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
69532025-11-12T20:48:35.278ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:38:35.278008763 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
69542025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
69552025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
69562025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 7 reason = Timeout
69572025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
69582025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
69592025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
69602025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69612025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
69622025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
69632025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
69642025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
69652025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
69662025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
69672025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
69682025-11-12T20:48:35.278ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
69692025-11-12T20:48:35.278ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = ExternalNetworking step id = Ensure
69702025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69712025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69722025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
69732025-11-12T20:48:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 772519513 } })
69742025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69752025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
69762025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
69772025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
69782025-11-12T20:48:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69792025-11-12T20:48:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 772762099 } })
69802025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 17 reason = Timeout
69812025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
69822025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
69832025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.304484ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
69842025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69852025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
69862025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
69872025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.577701ms iteration = 4 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
69882025-11-12T20:48:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69892025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69902025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
69912025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
69922025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
69932025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
69942025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69952025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69962025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69972025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
69982025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
69992025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
70002025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
70012025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 11 reason = Timeout
70022025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
70032025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
70042025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70052025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
70062025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
70072025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
70082025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
70092025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
70102025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70112025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 7 reason = Timeout
70122025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
70132025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70142025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
70152025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
70162025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
70172025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
70182025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
70192025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
70202025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
70212025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
70222025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
70232025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
70242025-11-12T20:48:35.280ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
70252025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70262025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
70272025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 37 reason = Timeout
70282025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70292025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70302025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
70312025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
70322025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
70332025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
70342025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
70352025-11-12T20:48:35.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
70362025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
70372025-11-12T20:48:35.281ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
70382025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
70392025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
70402025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
70412025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
70422025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
70432025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
70442025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70452025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70462025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
70472025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70482025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
70492025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70502025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
70512025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
70522025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70532025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
70542025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
70552025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70562025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
70572025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
70582025-11-12T20:48:35.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 775333486 } })
70592025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
70602025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
70612025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
70622025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70632025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
70642025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70652025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
70662025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70672025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70682025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
70692025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
70702025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
70712025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
70722025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
70732025-11-12T20:48:35.281ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70742025-11-12T20:48:35.282ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:38:35.278008763 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
70752025-11-12T20:48:35.282ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.298261ms iteration = 4 status = {"expiration":"2025-11-12T20:38:35.278008763Z","pruned":[]}
70762025-11-12T20:48:35.282ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70772025-11-12T20:48:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70782025-11-12T20:48:35.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70792025-11-12T20:48:35.282ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70802025-11-12T20:48:35.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70812025-11-12T20:48:35.283ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
70822025-11-12T20:48:35.283ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70832025-11-12T20:48:35.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
70842025-11-12T20:48:35.283ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70852025-11-12T20:48:35.284ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 778231331 } })
70862025-11-12T20:48:35.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70872025-11-12T20:48:35.284ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
70882025-11-12T20:48:35.284ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.870088ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
70892025-11-12T20:48:35.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70902025-11-12T20:48:35.285ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70912025-11-12T20:48:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70922025-11-12T20:48:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.924257ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
70932025-11-12T20:48:35.286ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70942025-11-12T20:48:35.286ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 780706605 } })
70952025-11-12T20:48:35.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
70962025-11-12T20:48:35.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
70972025-11-12T20:48:35.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 13 reason = Timeout
70982025-11-12T20:48:35.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
70992025-11-12T20:48:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
71002025-11-12T20:48:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
71012025-11-12T20:48:35.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71022025-11-12T20:48:35.289ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
71032025-11-12T20:48:35.289ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
71042025-11-12T20:48:35.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
71052025-11-12T20:48:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
71062025-11-12T20:48:35.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71072025-11-12T20:48:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71082025-11-12T20:48:35.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71092025-11-12T20:48:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71102025-11-12T20:48:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71112025-11-12T20:48:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
71122025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
71132025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
71142025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
71152025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
71162025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71172025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71182025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71192025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
71202025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71212025-11-12T20:48:35.292ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
71222025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 11.912949ms iteration = 4 status = {"by_rx":{},"error":null}
71232025-11-12T20:48:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71242025-11-12T20:48:35.293ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 4 reason = Timeout
71252025-11-12T20:48:35.293ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
71262025-11-12T20:48:35.293ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
71272025-11-12T20:48:35.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 13 reason = Timeout
71282025-11-12T20:48:35.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
71292025-11-12T20:48:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
71302025-11-12T20:48:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
71312025-11-12T20:48:35.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
71322025-11-12T20:48:35.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71332025-11-12T20:48:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71342025-11-12T20:48:35.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71352025-11-12T20:48:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
71362025-11-12T20:48:35.298ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71372025-11-12T20:48:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
71382025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71392025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71402025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
71412025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
71422025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
71432025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
71442025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
71452025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
71462025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
71472025-11-12T20:48:35.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
71482025-11-12T20:48:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
71492025-11-12T20:48:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
71502025-11-12T20:48:35.301ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71512025-11-12T20:48:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71522025-11-12T20:48:35.306ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71532025-11-12T20:48:35.306ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 26.142687ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
71542025-11-12T20:48:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71552025-11-12T20:48:35.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
71562025-11-12T20:48:35.307ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
71572025-11-12T20:48:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
71582025-11-12T20:48:35.307ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
71592025-11-12T20:48:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
71602025-11-12T20:48:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
71612025-11-12T20:48:35.310ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
71622025-11-12T20:48:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
71632025-11-12T20:48:35.310ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71642025-11-12T20:48:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71652025-11-12T20:48:35.314ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71662025-11-12T20:48:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71672025-11-12T20:48:35.314ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71682025-11-12T20:48:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
71692025-11-12T20:48:35.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
71702025-11-12T20:48:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
71712025-11-12T20:48:35.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71722025-11-12T20:48:35.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
71732025-11-12T20:48:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
71742025-11-12T20:48:35.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71752025-11-12T20:48:35.320ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71762025-11-12T20:48:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
71772025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71782025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71792025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4856956c-8555-4ed1-a4d2-383b97af5a6c original_time_started = 2025-11-12 20:47:36.587908 UTC
71802025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 42.471189ms iteration = 17 status = {"Loaded":{"collection_id":"4856956c-8555-4ed1-a4d2-383b97af5a6c","time_loaded":"2025-11-12T20:48:35.279058290Z","time_started":"2025-11-12T20:47:36.587908Z"}}
71812025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
71822025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
71832025-11-12T20:48:35.321ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
71842025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
71852025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
71862025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
71872025-11-12T20:48:35.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
71882025-11-12T20:48:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
71892025-11-12T20:48:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
71902025-11-12T20:48:35.323ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
71912025-11-12T20:48:35.323ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71922025-11-12T20:48:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
71932025-11-12T20:48:35.323ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
71942025-11-12T20:48:35.323ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 43.152066ms iteration = 37 status = {"config_updated":false}
71952025-11-12T20:48:35.324ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 37.350986ms iteration = 13 status = "NoSitrep"
71962025-11-12T20:48:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71972025-11-12T20:48:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
71982025-11-12T20:48:35.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71992025-11-12T20:48:35.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72002025-11-12T20:48:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72012025-11-12T20:48:35.327ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72022025-11-12T20:48:35.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72032025-11-12T20:48:35.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
72042025-11-12T20:48:35.327ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
72052025-11-12T20:48:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
72062025-11-12T20:48:35.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72072025-11-12T20:48:35.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72082025-11-12T20:48:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72092025-11-12T20:48:35.330ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 49.679985ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
72102025-11-12T20:48:35.330ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 50.071816ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
72112025-11-12T20:48:35.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72122025-11-12T20:48:35.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72132025-11-12T20:48:35.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72142025-11-12T20:48:35.333ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72152025-11-12T20:48:35.333ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72162025-11-12T20:48:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
72172025-11-12T20:48:35.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72182025-11-12T20:48:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
72192025-11-12T20:48:35.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
72202025-11-12T20:48:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72212025-11-12T20:48:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
72222025-11-12T20:48:35.335ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72232025-11-12T20:48:35.335ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
72242025-11-12T20:48:35.335ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 54.619123ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
72252025-11-12T20:48:35.336ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72262025-11-12T20:48:35.336ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
72272025-11-12T20:48:35.336ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 56.02729ms iteration = 7 status = 0
72282025-11-12T20:48:35.337ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72292025-11-12T20:48:35.337ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
72302025-11-12T20:48:35.337ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72312025-11-12T20:48:35.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72322025-11-12T20:48:35.338ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72332025-11-12T20:48:35.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72342025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
72352025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
72362025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
72372025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72382025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72392025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
72402025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72412025-11-12T20:48:35.340ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72422025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72432025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
72442025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
72452025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
72462025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72472025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
72482025-11-12T20:48:35.342ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72492025-11-12T20:48:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
72502025-11-12T20:48:35.343ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
72512025-11-12T20:48:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72522025-11-12T20:48:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
72532025-11-12T20:48:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
72542025-11-12T20:48:35.344ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72552025-11-12T20:48:35.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72562025-11-12T20:48:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
72572025-11-12T20:48:35.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72582025-11-12T20:48:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72592025-11-12T20:48:35.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72602025-11-12T20:48:35.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72612025-11-12T20:48:35.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72622025-11-12T20:48:35.355ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
72632025-11-12T20:48:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
72642025-11-12T20:48:35.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72652025-11-12T20:48:35.358ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72662025-11-12T20:48:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
72672025-11-12T20:48:35.360ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72682025-11-12T20:48:35.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72692025-11-12T20:48:35.363ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
72702025-11-12T20:48:35.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72712025-11-12T20:48:35.365ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
72722025-11-12T20:48:35.366ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 72.358502ms iteration = 13 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
72732025-11-12T20:48:35.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72742025-11-12T20:48:35.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72752025-11-12T20:48:35.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72762025-11-12T20:48:35.369ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 88.923102ms iteration = 3 status = {"nsuccess":5}
72772025-11-12T20:48:35.374ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72782025-11-12T20:48:35.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
72792025-11-12T20:48:35.377ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
72802025-11-12T20:48:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
72812025-11-12T20:48:35.378ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 97.355969ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
72822025-11-12T20:48:35.378ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72832025-11-12T20:48:35.379ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
72842025-11-12T20:48:35.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72852025-11-12T20:48:35.379ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
72862025-11-12T20:48:35.379ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
72872025-11-12T20:48:35.379ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
72882025-11-12T20:48:35.379ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 101.902377ms iteration = 4 status = {"generation":1}
72892025-11-12T20:48:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72902025-11-12T20:48:35.381ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
72912025-11-12T20:48:35.381ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
72922025-11-12T20:48:35.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
72932025-11-12T20:48:35.383ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72942025-11-12T20:48:35.383ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
72952025-11-12T20:48:35.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72962025-11-12T20:48:35.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72972025-11-12T20:48:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
72982025-11-12T20:48:35.387ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72992025-11-12T20:48:35.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73002025-11-12T20:48:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73012025-11-12T20:48:35.387ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
73022025-11-12T20:48:35.388ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
73032025-11-12T20:48:35.388ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
73042025-11-12T20:48:35.388ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 107.762889ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
73052025-11-12T20:48:35.389ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
73062025-11-12T20:48:35.389ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
73072025-11-12T20:48:35.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
73082025-11-12T20:48:35.390ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1110, tv_nsec: 884565013 } })
73092025-11-12T20:48:35.391ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
73102025-11-12T20:48:35.391ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
73112025-11-12T20:48:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73122025-11-12T20:48:35.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73132025-11-12T20:48:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73142025-11-12T20:48:35.393ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
73152025-11-12T20:48:35.394ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 114.049602ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
73162025-11-12T20:48:35.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73172025-11-12T20:48:35.395ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
73182025-11-12T20:48:35.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73192025-11-12T20:48:35.397ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
73202025-11-12T20:48:35.399ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73212025-11-12T20:48:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73222025-11-12T20:48:35.399ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
73232025-11-12T20:48:35.401ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73242025-11-12T20:48:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73252025-11-12T20:48:35.402ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73262025-11-12T20:48:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73272025-11-12T20:48:35.406ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
73282025-11-12T20:48:35.406ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 125.912499ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
73292025-11-12T20:48:35.406ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
73302025-11-12T20:48:35.409ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 128.149496ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73312025-11-12T20:48:35.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73322025-11-12T20:48:35.411ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
73332025-11-12T20:48:35.411ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
73342025-11-12T20:48:35.411ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
73352025-11-12T20:48:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 130.817666ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
73362025-11-12T20:48:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73372025-11-12T20:48:35.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73382025-11-12T20:48:35.416ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
73392025-11-12T20:48:35.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73402025-11-12T20:48:35.417ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = SledList step id = Fetch
73412025-11-12T20:48:35.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73422025-11-12T20:48:35.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73432025-11-12T20:48:35.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73442025-11-12T20:48:35.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73452025-11-12T20:48:35.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73462025-11-12T20:48:35.420ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73472025-11-12T20:48:35.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73482025-11-12T20:48:35.424ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
73492025-11-12T20:48:35.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73502025-11-12T20:48:35.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73512025-11-12T20:48:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
73522025-11-12T20:48:35.425ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73532025-11-12T20:48:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73542025-11-12T20:48:35.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73552025-11-12T20:48:35.428ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
73562025-11-12T20:48:35.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73572025-11-12T20:48:35.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73582025-11-12T20:48:35.428ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73592025-11-12T20:48:35.428ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
73602025-11-12T20:48:35.429ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
73612025-11-12T20:48:35.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73622025-11-12T20:48:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73632025-11-12T20:48:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73642025-11-12T20:48:35.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
73652025-11-12T20:48:35.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
73662025-11-12T20:48:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
73672025-11-12T20:48:35.430ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73682025-11-12T20:48:35.435ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
73692025-11-12T20:48:35.435ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
73702025-11-12T20:48:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
73712025-11-12T20:48:35.437ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
73722025-11-12T20:48:35.437ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
73732025-11-12T20:48:35.437ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
73742025-11-12T20:48:35.437ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 159.816446ms iteration = 4 status = {"generation":2}
73752025-11-12T20:48:35.442ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
73762025-11-12T20:48:35.442ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 162.131345ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
73772025-11-12T20:48:35.445ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
73782025-11-12T20:48:35.445ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 164.402185ms iteration = 19 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
73792025-11-12T20:48:35.457ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
73802025-11-12T20:48:35.469ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
73812025-11-12T20:48:35.469ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 191.076775ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
73822025-11-12T20:48:35.469ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 188.435027ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
73832025-11-12T20:48:35.474ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 193.58835ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
73842025-11-12T20:48:35.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73852025-11-12T20:48:35.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
73862025-11-12T20:48:35.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73872025-11-12T20:48:35.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73882025-11-12T20:48:35.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73892025-11-12T20:48:35.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73902025-11-12T20:48:35.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73912025-11-12T20:48:35.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73922025-11-12T20:48:35.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73932025-11-12T20:48:35.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73942025-11-12T20:48:35.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73952025-11-12T20:48:35.494ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
73962025-11-12T20:48:35.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73972025-11-12T20:48:35.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73982025-11-12T20:48:35.495ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73992025-11-12T20:48:35.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74002025-11-12T20:48:35.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74012025-11-12T20:48:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
74022025-11-12T20:48:35.498ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74032025-11-12T20:48:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74042025-11-12T20:48:35.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74052025-11-12T20:48:35.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
74062025-11-12T20:48:35.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74072025-11-12T20:48:35.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74082025-11-12T20:48:35.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74092025-11-12T20:48:35.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74102025-11-12T20:48:35.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74112025-11-12T20:48:35.514ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
74122025-11-12T20:48:35.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
74132025-11-12T20:48:35.515ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
74142025-11-12T20:48:35.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
74152025-11-12T20:48:35.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74162025-11-12T20:48:35.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74172025-11-12T20:48:35.536ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
74182025-11-12T20:48:35.536ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
74192025-11-12T20:48:35.537ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "688b967c-c1e4-46b1-9453-0de7530212e3", "content-length": "372", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
74202025-11-12T20:48:35.537ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74212025-11-12T20:48:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74222025-11-12T20:48:35.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74232025-11-12T20:48:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74242025-11-12T20:48:35.542ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
74252025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "37141696-ba4a-4069-b4fa-b12842e4213d", "content-length": "78", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
74262025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
74272025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
74282025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
74292025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
74302025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74312025-11-12T20:48:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
74322025-11-12T20:48:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
74332025-11-12T20:48:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74342025-11-12T20:48:35.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74352025-11-12T20:48:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74362025-11-12T20:48:35.545ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
74372025-11-12T20:48:35.545ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
74382025-11-12T20:48:35.545ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
74392025-11-12T20:48:35.546ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74402025-11-12T20:48:35.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74412025-11-12T20:48:35.550ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
74422025-11-12T20:48:35.550ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74432025-11-12T20:48:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74442025-11-12T20:48:35.551ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 270.476659ms iteration = 12 status = {"physical_disks_added":0}
74452025-11-12T20:48:35.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
74462025-11-12T20:48:35.552ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
74472025-11-12T20:48:35.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
74482025-11-12T20:48:35.552ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
74492025-11-12T20:48:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
74502025-11-12T20:48:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
74512025-11-12T20:48:35.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74522025-11-12T20:48:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
74532025-11-12T20:48:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74542025-11-12T20:48:35.554ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74552025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
74562025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
74572025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
74582025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
74592025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74602025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "13542554-f624-483e-8669-b73be4de00b5", "content-length": "81", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
74612025-11-12T20:48:35.555ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "932805fb-2ad5-441b-a134-b52b8212ad74", "content-length": "81", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
74622025-11-12T20:48:35.556ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
74632025-11-12T20:48:35.556ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
74642025-11-12T20:48:35.556ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = DeployNexusRecords step id = Ensure
74652025-11-12T20:48:35.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74662025-11-12T20:48:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
74672025-11-12T20:48:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74682025-11-12T20:48:35.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74692025-11-12T20:48:35.557ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
74702025-11-12T20:48:35.557ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74712025-11-12T20:48:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74722025-11-12T20:48:35.558ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
74732025-11-12T20:48:35.558ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
74742025-11-12T20:48:35.559ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f2e449c-4de4-4e8c-ad33-1ab2d78b035e", "content-length": "81", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
74752025-11-12T20:48:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
74762025-11-12T20:48:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74772025-11-12T20:48:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74782025-11-12T20:48:35.560ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
74792025-11-12T20:48:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74802025-11-12T20:48:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74812025-11-12T20:48:35.561ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 280.628462ms iteration = 7 status = {"disabled":false,"errors":[],"sps":[]}
74822025-11-12T20:48:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74832025-11-12T20:48:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
74842025-11-12T20:48:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74852025-11-12T20:48:35.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74862025-11-12T20:48:35.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74872025-11-12T20:48:35.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74882025-11-12T20:48:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74892025-11-12T20:48:35.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
74902025-11-12T20:48:35.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74912025-11-12T20:48:35.568ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
74922025-11-12T20:48:35.568ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74932025-11-12T20:48:35.569ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b67d475e-017d-44aa-83a5-0b09728ddf35", "content-length": "26", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
74942025-11-12T20:48:35.569ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74952025-11-12T20:48:35.570ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
74962025-11-12T20:48:35.571ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
74972025-11-12T20:48:35.573ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 295.549568ms iteration = 7 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
74982025-11-12T20:48:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
74992025-11-12T20:48:35.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75002025-11-12T20:48:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75012025-11-12T20:48:35.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75022025-11-12T20:48:35.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
75032025-11-12T20:48:35.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75042025-11-12T20:48:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75052025-11-12T20:48:35.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75062025-11-12T20:48:35.583ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
75072025-11-12T20:48:35.583ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
75082025-11-12T20:48:35.583ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "496ac79d-67f4-4e76-a926-bfe07c15932b", "content-length": "26", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
75092025-11-12T20:48:35.583ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
75102025-11-12T20:48:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75112025-11-12T20:48:35.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75122025-11-12T20:48:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75132025-11-12T20:48:35.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75142025-11-12T20:48:35.590ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
75152025-11-12T20:48:35.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
75162025-11-12T20:48:35.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75172025-11-12T20:48:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75182025-11-12T20:48:35.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75192025-11-12T20:48:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75202025-11-12T20:48:35.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75212025-11-12T20:48:35.603ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
75222025-11-12T20:48:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
75232025-11-12T20:48:35.604ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "6f211961-86d8-4a33-ab99-2e6ccc93e17b", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
75242025-11-12T20:48:35.604ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
75252025-11-12T20:48:35.606ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
75262025-11-12T20:48:35.606ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 312.875207ms iteration = 4 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
75272025-11-12T20:48:35.620ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
75282025-11-12T20:48:35.620ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
75292025-11-12T20:48:35.620ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2d538943-ecd0-4fe8-a701-8d74d2b9f4f5", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
75302025-11-12T20:48:35.620ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75312025-11-12T20:48:35.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75322025-11-12T20:48:35.623ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75332025-11-12T20:48:35.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75342025-11-12T20:48:35.627ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75352025-11-12T20:48:35.627ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = SledAgent step id = Ensure
75362025-11-12T20:48:35.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75372025-11-12T20:48:35.630ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
75382025-11-12T20:48:35.630ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
75392025-11-12T20:48:35.631ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "083e7404-9231-4e99-9725-eb3e7dcc9af4", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
75402025-11-12T20:48:35.633ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 352.371049ms iteration = 10 status = {}
75412025-11-12T20:48:35.633ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Dependency
75422025-11-12T20:48:35.633ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
75432025-11-12T20:48:35.633ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75442025-11-12T20:48:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75452025-11-12T20:48:35.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75462025-11-12T20:48:35.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75472025-11-12T20:48:35.638ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75482025-11-12T20:48:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75492025-11-12T20:48:35.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75502025-11-12T20:48:35.644ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
75512025-11-12T20:48:35.644ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75522025-11-12T20:48:35.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75532025-11-12T20:48:35.651ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
75542025-11-12T20:48:35.652ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c98de66a-e17c-402e-bcad-e49d214b9049", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
75552025-11-12T20:48:35.652ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
75562025-11-12T20:48:35.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "055d8da4-1b55-4b92-9b2e-3b39f6ec038d", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
75572025-11-12T20:48:35.656ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
75582025-11-12T20:48:35.659ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
75592025-11-12T20:48:35.659ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 380.715742ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
75602025-11-12T20:48:35.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75612025-11-12T20:48:35.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75622025-11-12T20:48:35.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75632025-11-12T20:48:35.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75642025-11-12T20:48:35.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75652025-11-12T20:48:35.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
75662025-11-12T20:48:35.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75672025-11-12T20:48:35.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75682025-11-12T20:48:35.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75692025-11-12T20:48:35.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75702025-11-12T20:48:35.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75712025-11-12T20:48:35.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
75722025-11-12T20:48:35.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75732025-11-12T20:48:35.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75742025-11-12T20:48:35.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75752025-11-12T20:48:35.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75762025-11-12T20:48:35.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75772025-11-12T20:48:35.695ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
75782025-11-12T20:48:35.695ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 415.297388ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
75792025-11-12T20:48:35.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
75802025-11-12T20:48:35.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75812025-11-12T20:48:35.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75822025-11-12T20:48:35.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75832025-11-12T20:48:35.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75842025-11-12T20:48:35.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75852025-11-12T20:48:35.702ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
75862025-11-12T20:48:35.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75872025-11-12T20:48:35.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75882025-11-12T20:48:35.705ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
75892025-11-12T20:48:35.705ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
75902025-11-12T20:48:35.705ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9207e14a-492f-4c88-9ef5-b2ded92aeb29 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
75912025-11-12T20:48:35.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75922025-11-12T20:48:35.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75932025-11-12T20:48:35.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75942025-11-12T20:48:35.709ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
75952025-11-12T20:48:35.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75962025-11-12T20:48:35.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75972025-11-12T20:48:35.713ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
75982025-11-12T20:48:35.713ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
75992025-11-12T20:48:35.713ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 432.723629ms iteration = 11 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
76002025-11-12T20:48:35.719ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
76012025-11-12T20:48:35.741ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 463.352671ms iteration = 7 status = {}
76022025-11-12T20:48:35.745ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
76032025-11-12T20:48:35.745ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "05017258-bb71-420d-9965-c0024b459b6b", "content-length": "691", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
76042025-11-12T20:48:35.745ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
76052025-11-12T20:48:35.745ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
76062025-11-12T20:48:35.745ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
76072025-11-12T20:48:35.749ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 468.873734ms iteration = 7 status = {}
76082025-11-12T20:48:35.758ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 252653868 } })
76092025-11-12T20:48:35.767ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
76102025-11-12T20:48:35.770ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "1a32cf86-70f8-4f9a-bdc5-f76695fdcf73", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
76112025-11-12T20:48:35.771ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = FirewallRules step id = Ensure
76122025-11-12T20:48:35.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76132025-11-12T20:48:35.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76142025-11-12T20:48:35.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76152025-11-12T20:48:35.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76162025-11-12T20:48:35.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76172025-11-12T20:48:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76182025-11-12T20:48:35.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76192025-11-12T20:48:35.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76202025-11-12T20:48:35.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76212025-11-12T20:48:35.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76222025-11-12T20:48:35.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76232025-11-12T20:48:35.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76242025-11-12T20:48:35.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76252025-11-12T20:48:35.788ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
76262025-11-12T20:48:35.788ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
76272025-11-12T20:48:35.788ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "55df0fb7-5de8-40bf-bb1e-49a708e3e59e", "content-length": "372", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
76282025-11-12T20:48:35.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76292025-11-12T20:48:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76302025-11-12T20:48:35.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
76312025-11-12T20:48:35.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76322025-11-12T20:48:35.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76332025-11-12T20:48:35.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76342025-11-12T20:48:35.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76352025-11-12T20:48:35.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76362025-11-12T20:48:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76372025-11-12T20:48:35.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76382025-11-12T20:48:35.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
76392025-11-12T20:48:35.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76402025-11-12T20:48:35.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76412025-11-12T20:48:35.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76422025-11-12T20:48:35.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76432025-11-12T20:48:35.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76442025-11-12T20:48:35.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76452025-11-12T20:48:35.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76462025-11-12T20:48:35.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76472025-11-12T20:48:35.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76482025-11-12T20:48:35.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76492025-11-12T20:48:35.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76502025-11-12T20:48:35.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76512025-11-12T20:48:35.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76522025-11-12T20:48:35.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76532025-11-12T20:48:35.815ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
76542025-11-12T20:48:35.815ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ec6df461-ee16-4abb-b9f7-0f6376263545", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76552025-11-12T20:48:35.815ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76562025-11-12T20:48:35.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76572025-11-12T20:48:35.817ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "69b7b39c-7e30-43ea-a7fb-f3203cd55cb6", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76582025-11-12T20:48:35.817ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76592025-11-12T20:48:35.819ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "eca0e8c5-5fe8-49a5-b0ad-adf290bbf1d0", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76602025-11-12T20:48:35.819ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76612025-11-12T20:48:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
76622025-11-12T20:48:35.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76632025-11-12T20:48:35.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76642025-11-12T20:48:35.821ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
76652025-11-12T20:48:35.821ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 543.814733ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
76662025-11-12T20:48:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76672025-11-12T20:48:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76682025-11-12T20:48:35.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76692025-11-12T20:48:35.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76702025-11-12T20:48:35.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76712025-11-12T20:48:35.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76722025-11-12T20:48:35.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76732025-11-12T20:48:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76742025-11-12T20:48:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76752025-11-12T20:48:35.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76762025-11-12T20:48:35.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76772025-11-12T20:48:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76782025-11-12T20:48:35.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76792025-11-12T20:48:35.845ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
76802025-11-12T20:48:35.845ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
76812025-11-12T20:48:35.846ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "196ca8b1-0c5a-4cc3-825e-d42ba62afc53", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
76822025-11-12T20:48:35.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76832025-11-12T20:48:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76842025-11-12T20:48:35.848ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
76852025-11-12T20:48:35.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76862025-11-12T20:48:35.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76872025-11-12T20:48:35.848ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76882025-11-12T20:48:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76892025-11-12T20:48:35.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76902025-11-12T20:48:35.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76912025-11-12T20:48:35.851ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76922025-11-12T20:48:35.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
76932025-11-12T20:48:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76942025-11-12T20:48:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
76952025-11-12T20:48:35.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76962025-11-12T20:48:35.856ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
76972025-11-12T20:48:35.856ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
76982025-11-12T20:48:35.856ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c3c70f2a-9e4a-4eb2-83bc-2381c9d82ba4", "date": "Wed, 12 Nov 2025 20:48:35 GMT"} })
76992025-11-12T20:48:35.857ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
77002025-11-12T20:48:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77012025-11-12T20:48:35.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77022025-11-12T20:48:35.858ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 225.607228ms iteration = 11 status = {}
77032025-11-12T20:48:35.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77042025-11-12T20:48:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77052025-11-12T20:48:35.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77062025-11-12T20:48:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77072025-11-12T20:48:35.863ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
77082025-11-12T20:48:35.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77092025-11-12T20:48:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77102025-11-12T20:48:35.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77112025-11-12T20:48:35.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77122025-11-12T20:48:35.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77132025-11-12T20:48:35.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77142025-11-12T20:48:35.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77152025-11-12T20:48:35.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77162025-11-12T20:48:35.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77172025-11-12T20:48:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77182025-11-12T20:48:35.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77192025-11-12T20:48:35.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77202025-11-12T20:48:35.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77212025-11-12T20:48:35.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77222025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
77232025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
77242025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
77252025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
77262025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77272025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77282025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77292025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77302025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77312025-11-12T20:48:35.874ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77322025-11-12T20:48:35.875ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
77332025-11-12T20:48:35.875ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77342025-11-12T20:48:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77352025-11-12T20:48:35.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77362025-11-12T20:48:35.877ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
77372025-11-12T20:48:35.877ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
77382025-11-12T20:48:35.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
77392025-11-12T20:48:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77402025-11-12T20:48:35.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77412025-11-12T20:48:35.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77422025-11-12T20:48:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77432025-11-12T20:48:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77442025-11-12T20:48:35.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77452025-11-12T20:48:35.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77462025-11-12T20:48:35.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
77472025-11-12T20:48:35.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77482025-11-12T20:48:35.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77492025-11-12T20:48:35.897ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
77502025-11-12T20:48:35.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77512025-11-12T20:48:35.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77522025-11-12T20:48:35.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77532025-11-12T20:48:35.900ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
77542025-11-12T20:48:35.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77552025-11-12T20:48:35.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77562025-11-12T20:48:35.903ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
77572025-11-12T20:48:35.903ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
77582025-11-12T20:48:35.903ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
77592025-11-12T20:48:35.974ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
77602025-11-12T20:48:35.974ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
77612025-11-12T20:48:36.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77622025-11-12T20:48:36.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77632025-11-12T20:48:36.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77642025-11-12T20:48:36.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77652025-11-12T20:48:36.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77662025-11-12T20:48:36.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
77672025-11-12T20:48:36.013ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
77682025-11-12T20:48:36.014ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "88f526b4-caf3-4848-adf1-34c5d36b35d5", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
77692025-11-12T20:48:36.015ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
77702025-11-12T20:48:36.015ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = Dns step id = Ensure
77712025-11-12T20:48:36.015ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
77722025-11-12T20:48:36.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77732025-11-12T20:48:36.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77742025-11-12T20:48:36.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77752025-11-12T20:48:36.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77762025-11-12T20:48:36.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77772025-11-12T20:48:36.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
77782025-11-12T20:48:36.023ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
77792025-11-12T20:48:36.023ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
77802025-11-12T20:48:36.023ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
77812025-11-12T20:48:36.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77822025-11-12T20:48:36.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77832025-11-12T20:48:36.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77842025-11-12T20:48:36.028ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
77852025-11-12T20:48:36.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77862025-11-12T20:48:36.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77872025-11-12T20:48:36.030ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
77882025-11-12T20:48:36.030ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
77892025-11-12T20:48:36.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77902025-11-12T20:48:36.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77912025-11-12T20:48:36.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77922025-11-12T20:48:36.035ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
77932025-11-12T20:48:36.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77942025-11-12T20:48:36.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
77952025-11-12T20:48:36.037ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
77962025-11-12T20:48:36.037ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
77972025-11-12T20:48:36.037ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
77982025-11-12T20:48:36.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
77992025-11-12T20:48:36.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78002025-11-12T20:48:36.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78012025-11-12T20:48:36.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78022025-11-12T20:48:36.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78032025-11-12T20:48:36.043ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
78042025-11-12T20:48:36.044ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
78052025-11-12T20:48:36.044ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
78062025-11-12T20:48:36.044ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
78072025-11-12T20:48:36.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78082025-11-12T20:48:36.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78092025-11-12T20:48:36.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78102025-11-12T20:48:36.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
78112025-11-12T20:48:36.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78122025-11-12T20:48:36.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78132025-11-12T20:48:36.051ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
78142025-11-12T20:48:36.051ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
78152025-11-12T20:48:36.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78162025-11-12T20:48:36.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78172025-11-12T20:48:36.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78182025-11-12T20:48:36.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
78192025-11-12T20:48:36.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78202025-11-12T20:48:36.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78212025-11-12T20:48:36.057ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
78222025-11-12T20:48:36.057ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
78232025-11-12T20:48:36.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78242025-11-12T20:48:36.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78252025-11-12T20:48:36.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78262025-11-12T20:48:36.061ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
78272025-11-12T20:48:36.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78282025-11-12T20:48:36.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78292025-11-12T20:48:36.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78302025-11-12T20:48:36.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78312025-11-12T20:48:36.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78322025-11-12T20:48:36.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78332025-11-12T20:48:36.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78342025-11-12T20:48:36.069ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
78352025-11-12T20:48:36.071ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
78362025-11-12T20:48:36.071ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
78372025-11-12T20:48:36.071ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = OmicronZones step id = Cleanup
78382025-11-12T20:48:36.071ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = OmicronZones step id = Cleanup
78392025-11-12T20:48:36.071ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = PhysicalDisks step id = Cleanup
78402025-11-12T20:48:36.071ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = Clickhouse step id = Ensure
78412025-11-12T20:48:36.071ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = Clickhouse step id = Ensure
78422025-11-12T20:48:36.107ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
78432025-11-12T20:48:36.112ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "e0467510-4e50-42f6-a7a7-b95b5b11c47d", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
78442025-11-12T20:48:36.114ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = SupportBundles step id = Cleanup
78452025-11-12T20:48:36.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78462025-11-12T20:48:36.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78472025-11-12T20:48:36.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78482025-11-12T20:48:36.117ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b03ff917-839a-45f3-9ffd-2a2e01c97e56", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
78492025-11-12T20:48:36.117ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
78502025-11-12T20:48:36.117ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
78512025-11-12T20:48:36.118ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
78522025-11-12T20:48:36.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78532025-11-12T20:48:36.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78542025-11-12T20:48:36.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78552025-11-12T20:48:36.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78562025-11-12T20:48:36.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78572025-11-12T20:48:36.124ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "73ff8101-e7f6-4dec-9993-862da056479a", "content-length": "682", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
78582025-11-12T20:48:36.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78592025-11-12T20:48:36.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78602025-11-12T20:48:36.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
78612025-11-12T20:48:36.127ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78622025-11-12T20:48:36.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78632025-11-12T20:48:36.128ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = OmicronZones step id = Cleanup
78642025-11-12T20:48:36.128ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
78652025-11-12T20:48:36.128ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
78662025-11-12T20:48:36.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78672025-11-12T20:48:36.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78682025-11-12T20:48:36.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78692025-11-12T20:48:36.131ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78702025-11-12T20:48:36.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78712025-11-12T20:48:36.132ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
78722025-11-12T20:48:36.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78732025-11-12T20:48:36.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78742025-11-12T20:48:36.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78752025-11-12T20:48:36.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78762025-11-12T20:48:36.140ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 860.283003ms iteration = 7 status = {}
78772025-11-12T20:48:36.171ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
78782025-11-12T20:48:36.171ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
78792025-11-12T20:48:36.171ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = Cockroach step id = Ensure
78802025-11-12T20:48:36.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78812025-11-12T20:48:36.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78822025-11-12T20:48:36.182ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
78832025-11-12T20:48:36.182ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1e7391b1-906a-4f72-9a3d-1dab4229432d step component = MgsUpdates step id = Ensure
78842025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Dependency
78852025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
78862025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Dependency
78872025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
78882025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 904.27266ms iteration = 5 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3290,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":139357267,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":139357217,"secs":0}},"event_index":1,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":139372177,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":138624617,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":138624568,"secs":0}},"event_index":2,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":278010854,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":71012227,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":71012187,"secs":0}},"event_index":3,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":349037263,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":144651713,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":144651673,"secs":0}},"event_index":4,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":493699516,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":244033165,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":244033115,"secs":0}},"event_index":5,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":737742151,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":55509937,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":55509897,"secs":0}},"event_index":6,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":793261078,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6200,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":6160,"secs":0}},"event_index":7,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":793282348,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2930,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2900,"secs":0}},"event_index":8,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":793288598,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3130,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3101,"secs":0}},"event_index":9,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":793293879,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1680,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1640,"secs":0}},"event_index":10,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":793302139,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":42918920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":42918880,"secs":0}},"event_index":11,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":836222359,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14239878,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14239838,"secs":0}},"event_index":12,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":850478058,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":42517880,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":42517840,"secs":0}},"event_index":13,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":893010319,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":10968274,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":10968234,"secs":0}},"event_index":14,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903988203,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5561,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5530,"secs":0}},"event_index":15,"execution_id":"d2c16190-8e2e-45d1-a210-5916298ff7cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":904004484,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
78892025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
78902025-11-12T20:48:36.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
78912025-11-12T20:48:36.184ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78922025-11-12T20:48:36.184ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78932025-11-12T20:48:36.187ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
78942025-11-12T20:48:36.187ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
78952025-11-12T20:48:36.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
78962025-11-12T20:48:36.187ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
78972025-11-12T20:48:36.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
78982025-11-12T20:48:36.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
78992025-11-12T20:48:36.189ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
79002025-11-12T20:48:36.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
79012025-11-12T20:48:36.191ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79022025-11-12T20:48:36.191ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(d0500078-19af-4275-a420-eb4354ebb2e5, 0), (1b79856a-d0c9-4661-821f-cb877e3669c1, 0), (be6f941b-6853-4193-8c1b-4c65d2be0df1, 0), (c6bdc9e0-c078-4e8f-92d4-c421719a1051, 0)] collection_id = d0500078-19af-4275-a420-eb4354ebb2e5
79032025-11-12T20:48:36.191ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79042025-11-12T20:48:36.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79052025-11-12T20:48:36.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
79062025-11-12T20:48:36.193ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
79072025-11-12T20:48:36.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
79082025-11-12T20:48:36.195ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
79092025-11-12T20:48:36.195ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 13.505549ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
79102025-11-12T20:48:36.362ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = d0500078-19af-4275-a420-eb4354ebb2e5 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
79112025-11-12T20:48:36.362ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79122025-11-12T20:48:36.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79132025-11-12T20:48:36.366ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(1b79856a-d0c9-4661-821f-cb877e3669c1, 0), (be6f941b-6853-4193-8c1b-4c65d2be0df1, 0), (c6bdc9e0-c078-4e8f-92d4-c421719a1051, 0), (677c0ea6-1049-4589-9f51-940941493fc6, 0)] collection_id = 1b79856a-d0c9-4661-821f-cb877e3669c1
79142025-11-12T20:48:36.366ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79152025-11-12T20:48:36.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79162025-11-12T20:48:36.568ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 1b79856a-d0c9-4661-821f-cb877e3669c1 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
79172025-11-12T20:48:36.568ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79182025-11-12T20:48:36.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79192025-11-12T20:48:36.691ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(c6bdc9e0-c078-4e8f-92d4-c421719a1051, 0), (677c0ea6-1049-4589-9f51-940941493fc6, 0), (4856956c-8555-4ed1-a4d2-383b97af5a6c, 0)]
79202025-11-12T20:48:36.691ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1112, tv_nsec: 185747225 } })
79212025-11-12T20:48:36.723ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1112, tv_nsec: 217342203 } })
79222025-11-12T20:48:36.723ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
79232025-11-12T20:48:36.724ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
79242025-11-12T20:48:36.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
79252025-11-12T20:48:36.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f76fe36-565e-4258-8341-e5006b460b70", "content-length": "476", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79262025-11-12T20:48:36.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
79272025-11-12T20:48:36.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd18eed2-a02b-46bf-88f0-aef449933590", "content-length": "734", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79282025-11-12T20:48:36.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
79292025-11-12T20:48:36.725ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfbda281-7952-4ad4-85ca-fefd05ebe70b", "content-length": "111", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79302025-11-12T20:48:36.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
79312025-11-12T20:48:36.725ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "422df5c3-768b-445f-b42e-1f9c95c29596", "content-length": "111", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79322025-11-12T20:48:36.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
79332025-11-12T20:48:36.726ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db329088-c99c-4cbd-87bb-42c3f432fe52", "content-length": "163", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79342025-11-12T20:48:36.726ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
79352025-11-12T20:48:36.726ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b2d07b-0bf8-42e8-86cf-b465ae5b5125", "content-length": "163", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79362025-11-12T20:48:36.726ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
79372025-11-12T20:48:36.726ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "28aa473a-2be1-4b7d-886c-e985a8afda7f", "content-length": "166", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79382025-11-12T20:48:36.726ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
79392025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3177ed52-be4f-4e09-899f-b519b757af54", "content-length": "166", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79402025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
79412025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "40265c84-6b0d-43cf-ba14-6ba733c92520", "content-length": "702", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79422025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
79432025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfb8774e-ae94-43d7-a6cf-a66a4e1d80f9", "content-length": "727", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79442025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
79452025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e2ff066-5d92-4e46-be5f-5fdb3738d475", "content-length": "729", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79462025-11-12T20:48:36.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
79472025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fd4b891-e29d-4d1d-932c-71f59156fc8b", "content-length": "728", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79482025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
79492025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "077c1d24-1fbd-42ec-a577-8d8d5e42aa2e", "content-length": "722", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79502025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
79512025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "24c6f46f-9c91-4fef-b57a-45a0434163f3", "content-length": "10", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79522025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
79532025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a72de5b-7ead-416d-aa12-ce2f68585d65", "content-length": "144", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79542025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
79552025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5b841be-18c9-491c-a824-e3829d5e5cb8", "content-length": "147", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79562025-11-12T20:48:36.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
79572025-11-12T20:48:36.729ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f4f5e94-486d-4a40-9e6d-b2062649f8c7", "content-length": "109", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79582025-11-12T20:48:36.729ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
79592025-11-12T20:48:36.729ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b80272a-1cc9-4440-a8f9-c3d173a23f19", "content-length": "109", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79602025-11-12T20:48:36.729ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
79612025-11-12T20:48:36.730ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "befaae62-486d-4b33-ae12-c22f8c76d461", "content-length": "163", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79622025-11-12T20:48:36.730ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
79632025-11-12T20:48:36.730ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0add85b-762d-45ca-8fe6-9d7ec3b7ab0e", "content-length": "163", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79642025-11-12T20:48:36.730ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
79652025-11-12T20:48:36.730ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1dda098-46de-4fd8-a0b2-bcb1ca0bb7e4", "content-length": "166", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79662025-11-12T20:48:36.730ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
79672025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68706131-6968-48d0-a30b-b8d799e908b5", "content-length": "166", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79682025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
79692025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6d9cf79-578f-4a3f-9b0f-1392202eac46", "content-length": "702", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79702025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
79712025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52cc89ee-3b81-474a-ae07-8bd7f292ae33", "content-length": "727", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79722025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
79732025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a8423d3-3b83-43e3-b485-b9b0d26f5a4b", "content-length": "729", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79742025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
79752025-11-12T20:48:36.731ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9924ed9-0e2d-4231-ab04-db8d0c65beb3", "content-length": "728", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79762025-11-12T20:48:36.732ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79772025-11-12T20:48:36.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79782025-11-12T20:48:36.734ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79792025-11-12T20:48:36.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79802025-11-12T20:48:36.736ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79812025-11-12T20:48:36.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79822025-11-12T20:48:36.738ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79832025-11-12T20:48:36.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79842025-11-12T20:48:36.740ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79852025-11-12T20:48:36.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
79862025-11-12T20:48:36.777ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
79872025-11-12T20:48:36.778ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
79882025-11-12T20:48:36.867ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ec114b6-9724-4668-9637-888fb08d5dcb", "content-length": "95175", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79892025-11-12T20:48:36.869ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79902025-11-12T20:48:36.869ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79912025-11-12T20:48:36.869ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
79922025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
79932025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
79942025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
79952025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
79962025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
79972025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "43211edb-3b76-4ef4-8199-fc83dd3557c8", "content-length": "64", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79982025-11-12T20:48:36.869ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2db0e58-26fc-429c-84bd-4780fcb576a3", "content-length": "64", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
79992025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
80002025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec755935-61a6-46f3-8722-506f41717d20", "content-length": "64", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80012025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "630c0859-6067-495b-9d58-0d543d2d1d5d", "content-length": "64", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80022025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
80032025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
80042025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
80052025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6f3f0d0-b924-4df4-ab40-d0132fe18f57", "content-length": "64", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80062025-11-12T20:48:36.870ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
80072025-11-12T20:48:36.910ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "29e20119-e05b-46a3-904e-1d0496a4f34f", "content-length": "385924", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80082025-11-12T20:48:36.910ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2417786-4870-40a4-b431-483ec76fb82b", "content-length": "386018", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80092025-11-12T20:48:36.910ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eb0873f-c91a-4569-b6fd-52d9610bd107", "content-length": "390726", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80102025-11-12T20:48:36.911ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d3c62d5-0037-412a-8539-009cb9463309", "content-length": "418873", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80112025-11-12T20:48:36.913ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "13797565-d078-4a78-9d1d-8dd05cd79ea6", "content-length": "381742", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80122025-11-12T20:48:36.916ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
80132025-11-12T20:48:36.916ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
80142025-11-12T20:48:36.916ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
80152025-11-12T20:48:36.916ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
80162025-11-12T20:48:36.916ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
80172025-11-12T20:48:36.947ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
80182025-11-12T20:48:36.947ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
80192025-11-12T20:48:36.950ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "684192b1-3ec2-49bc-826f-afda8debde80", "content-length": "124", "date": "Wed, 12 Nov 2025 20:48:36 GMT"} })
80202025-11-12T20:48:36.951ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
80212025-11-12T20:48:37.042ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
80222025-11-12T20:48:37.042ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
80232025-11-12T20:48:37.043ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8c2e085-b169-434a-90b5-27171ff0d298", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:48:37 GMT"} })
80242025-11-12T20:48:37.043ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
80252025-11-12T20:48:37.044ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
80262025-11-12T20:48:37.044ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
80272025-11-12T20:48:37.045ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e3d4586-7770-4810-af96-0973c608dd31", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:48:37 GMT"} })
80282025-11-12T20:48:37.045ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
80292025-11-12T20:48:37.047ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
80302025-11-12T20:48:37.047ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
80312025-11-12T20:48:37.048ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "45e3bcaf-4ee4-40c9-b5f2-16b2e6cc50c3", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:48:37 GMT"} })
80322025-11-12T20:48:37.048ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
80332025-11-12T20:48:37.049ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
80342025-11-12T20:48:37.049ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
80352025-11-12T20:48:37.051ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
80362025-11-12T20:48:37.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
80372025-11-12T20:48:37.053ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80382025-11-12T20:48:37.055ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
80392025-11-12T20:48:37.056ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
80402025-11-12T20:48:37.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
80412025-11-12T20:48:37.264ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 17566f7a-6076-4f56-b7da-c6a2f3173ae2 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
80422025-11-12T20:48:37.264ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 17566f7a-6076-4f56-b7da-c6a2f3173ae2 time_started = 2025-11-12 20:48:36.723978 UTC
80432025-11-12T20:48:37.264ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 18 reason = Dependency
80442025-11-12T20:48:37.264ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
80452025-11-12T20:48:37.264ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.08199312s iteration = 5 status = {"collection_id":"17566f7a-6076-4f56-b7da-c6a2f3173ae2","time_done":"2025-11-12 20:48:37.049843 UTC","time_started":"2025-11-12 20:48:36.723978 UTC"}
80462025-11-12T20:48:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
80472025-11-12T20:48:37.266ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80482025-11-12T20:48:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
80492025-11-12T20:48:37.269ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
80502025-11-12T20:48:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
80512025-11-12T20:48:37.353ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
80522025-11-12T20:48:37.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
80532025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 138.297979ms iteration = 18 status = {"Loaded":{"collection_id":"49827f1a-3808-4b86-9622-c369fde9d0c0","time_loaded":"2025-11-12T20:48:37.264182941Z","time_started":"2025-11-12T20:48:36.725448Z"}}
80542025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 12 reason = Dependency
80552025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Dependency
80562025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 7 reason = Dependency
80572025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80582025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80592025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
80602025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80612025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
80622025-11-12T20:48:37.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80632025-11-12T20:48:37.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80642025-11-12T20:48:37.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80652025-11-12T20:48:37.404ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80662025-11-12T20:48:37.407ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
80672025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80682025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
80692025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
80702025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80712025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
80722025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80732025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
80742025-11-12T20:48:37.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80752025-11-12T20:48:37.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80762025-11-12T20:48:37.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80772025-11-12T20:48:37.410ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80782025-11-12T20:48:37.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80792025-11-12T20:48:37.412ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80802025-11-12T20:48:37.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
80812025-11-12T20:48:37.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80822025-11-12T20:48:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80832025-11-12T20:48:37.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
80842025-11-12T20:48:37.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
80852025-11-12T20:48:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
80862025-11-12T20:48:37.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
80872025-11-12T20:48:37.415ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80882025-11-12T20:48:37.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80892025-11-12T20:48:37.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80902025-11-12T20:48:37.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80912025-11-12T20:48:37.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80922025-11-12T20:48:37.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
80932025-11-12T20:48:37.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80942025-11-12T20:48:37.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80952025-11-12T20:48:37.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80962025-11-12T20:48:37.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
80972025-11-12T20:48:37.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
80982025-11-12T20:48:37.426ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
80992025-11-12T20:48:37.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81002025-11-12T20:48:37.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81012025-11-12T20:48:37.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81022025-11-12T20:48:37.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81032025-11-12T20:48:37.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81042025-11-12T20:48:37.431ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
81052025-11-12T20:48:37.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
81062025-11-12T20:48:37.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
81072025-11-12T20:48:37.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81082025-11-12T20:48:37.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81092025-11-12T20:48:37.433ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 31.403992ms iteration = 13 status = {"physical_disks_added":0}
81102025-11-12T20:48:37.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81112025-11-12T20:48:37.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
81122025-11-12T20:48:37.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
81132025-11-12T20:48:37.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81142025-11-12T20:48:37.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
81152025-11-12T20:48:37.438ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
81162025-11-12T20:48:37.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
81172025-11-12T20:48:37.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
81182025-11-12T20:48:37.439ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81192025-11-12T20:48:37.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81202025-11-12T20:48:37.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81212025-11-12T20:48:37.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81222025-11-12T20:48:37.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81232025-11-12T20:48:37.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81242025-11-12T20:48:37.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
81252025-11-12T20:48:37.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
81262025-11-12T20:48:37.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
81272025-11-12T20:48:37.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81282025-11-12T20:48:37.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81292025-11-12T20:48:37.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
81302025-11-12T20:48:37.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81312025-11-12T20:48:37.446ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
81322025-11-12T20:48:37.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
81332025-11-12T20:48:37.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81342025-11-12T20:48:37.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
81352025-11-12T20:48:37.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81362025-11-12T20:48:37.449ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
81372025-11-12T20:48:37.449ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 46.7708ms iteration = 7 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"49827f1a-3808-4b86-9622-c369fde9d0c0","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
81382025-11-12T20:48:37.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81392025-11-12T20:48:37.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81402025-11-12T20:48:37.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81412025-11-12T20:48:37.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81422025-11-12T20:48:37.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81432025-11-12T20:48:37.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81442025-11-12T20:48:37.458ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
81452025-11-12T20:48:37.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81462025-11-12T20:48:37.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81472025-11-12T20:48:37.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81482025-11-12T20:48:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81492025-11-12T20:48:37.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81502025-11-12T20:48:37.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
81512025-11-12T20:48:37.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81522025-11-12T20:48:37.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81532025-11-12T20:48:37.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81542025-11-12T20:48:37.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81552025-11-12T20:48:37.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81562025-11-12T20:48:37.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
81572025-11-12T20:48:37.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81582025-11-12T20:48:37.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81592025-11-12T20:48:37.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81602025-11-12T20:48:37.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81612025-11-12T20:48:37.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81622025-11-12T20:48:37.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81632025-11-12T20:48:37.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81642025-11-12T20:48:37.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81652025-11-12T20:48:37.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81662025-11-12T20:48:37.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81672025-11-12T20:48:37.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81682025-11-12T20:48:37.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81692025-11-12T20:48:37.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81702025-11-12T20:48:37.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81712025-11-12T20:48:37.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81722025-11-12T20:48:37.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81732025-11-12T20:48:37.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81742025-11-12T20:48:37.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81752025-11-12T20:48:37.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81762025-11-12T20:48:37.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81772025-11-12T20:48:37.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81782025-11-12T20:48:37.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81792025-11-12T20:48:37.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81802025-11-12T20:48:37.513ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81812025-11-12T20:48:37.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81822025-11-12T20:48:37.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81832025-11-12T20:48:37.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81842025-11-12T20:48:37.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81852025-11-12T20:48:37.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81862025-11-12T20:48:37.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81872025-11-12T20:48:37.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81882025-11-12T20:48:37.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
81892025-11-12T20:48:37.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81902025-11-12T20:48:37.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81912025-11-12T20:48:37.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81922025-11-12T20:48:37.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81932025-11-12T20:48:37.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81942025-11-12T20:48:37.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
81952025-11-12T20:48:37.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81962025-11-12T20:48:37.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81972025-11-12T20:48:37.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81982025-11-12T20:48:37.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81992025-11-12T20:48:37.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82002025-11-12T20:48:37.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
82012025-11-12T20:48:37.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82022025-11-12T20:48:37.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82032025-11-12T20:48:37.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82042025-11-12T20:48:37.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82052025-11-12T20:48:37.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82062025-11-12T20:48:37.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
82072025-11-12T20:48:37.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82082025-11-12T20:48:37.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82092025-11-12T20:48:37.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82102025-11-12T20:48:37.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82112025-11-12T20:48:37.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82122025-11-12T20:48:37.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
82132025-11-12T20:48:37.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82142025-11-12T20:48:37.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82152025-11-12T20:48:37.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82162025-11-12T20:48:37.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82172025-11-12T20:48:37.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82182025-11-12T20:48:37.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
82192025-11-12T20:48:37.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82202025-11-12T20:48:37.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82212025-11-12T20:48:37.590ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
82222025-11-12T20:48:37.590ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
82232025-11-12T20:48:37.590ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = c5e7e7bc-a738-4e41-88dd-72078206adaf parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
82242025-11-12T20:48:37.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82252025-11-12T20:48:37.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82262025-11-12T20:48:37.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82272025-11-12T20:48:37.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
82282025-11-12T20:48:37.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82292025-11-12T20:48:37.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82302025-11-12T20:48:37.598ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
82312025-11-12T20:48:37.598ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
82322025-11-12T20:48:37.598ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 196.027353ms iteration = 12 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
82332025-11-12T20:48:37.941ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 337 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 100a3abd-02d2-40b7-83c9-f3249f87890a response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
82342025-11-12T20:48:40.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 38 reason = Timeout
82352025-11-12T20:48:40.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82362025-11-12T20:48:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82372025-11-12T20:48:40.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82382025-11-12T20:48:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
82392025-11-12T20:48:40.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82402025-11-12T20:48:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82412025-11-12T20:48:40.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
82422025-11-12T20:48:40.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.452197ms iteration = 38 status = {"config_updated":false}
82432025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
82442025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82452025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
82462025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82472025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 39 reason = Timeout
82482025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82492025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82502025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82512025-11-12T20:48:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
82522025-11-12T20:48:45.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82532025-11-12T20:48:45.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82542025-11-12T20:48:45.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82552025-11-12T20:48:45.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82562025-11-12T20:48:45.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.326692ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
82572025-11-12T20:48:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
82582025-11-12T20:48:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82592025-11-12T20:48:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
82602025-11-12T20:48:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
82612025-11-12T20:48:45.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82622025-11-12T20:48:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82632025-11-12T20:48:45.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82642025-11-12T20:48:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
82652025-11-12T20:48:45.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
82662025-11-12T20:48:45.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.280964ms iteration = 39 status = {"config_updated":false}
82672025-11-12T20:48:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82682025-11-12T20:48:45.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
82692025-11-12T20:48:45.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82702025-11-12T20:48:45.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
82712025-11-12T20:48:45.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
82722025-11-12T20:48:45.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.450413ms iteration = 20 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
82732025-11-12T20:48:47.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 294 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 78eef2f3-a9aa-4359-baa2-760e4a7037f1 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
82742025-11-12T20:48:50.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 19 reason = Timeout
82752025-11-12T20:48:50.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
82762025-11-12T20:48:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
82772025-11-12T20:48:50.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 40 reason = Timeout
82782025-11-12T20:48:50.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82792025-11-12T20:48:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82802025-11-12T20:48:50.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82812025-11-12T20:48:50.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82822025-11-12T20:48:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
82832025-11-12T20:48:50.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
82842025-11-12T20:48:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82852025-11-12T20:48:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
82862025-11-12T20:48:50.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
82872025-11-12T20:48:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
82882025-11-12T20:48:50.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
82892025-11-12T20:48:50.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.594161ms iteration = 40 status = {"config_updated":false}
82902025-11-12T20:48:50.287ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49827f1a-3808-4b86-9622-c369fde9d0c0 original_time_started = 2025-11-12 20:48:36.725448 UTC
82912025-11-12T20:48:50.287ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.831703ms iteration = 19 status = {"Loaded":{"collection_id":"49827f1a-3808-4b86-9622-c369fde9d0c0","time_loaded":"2025-11-12T20:48:50.279889575Z","time_started":"2025-11-12T20:48:36.725448Z"}}
82922025-11-12T20:48:50.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 14 reason = Timeout
82932025-11-12T20:48:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
82942025-11-12T20:48:50.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
82952025-11-12T20:48:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82962025-11-12T20:48:50.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
82972025-11-12T20:48:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
82982025-11-12T20:48:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
82992025-11-12T20:48:50.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 14 reason = Timeout
83002025-11-12T20:48:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
83012025-11-12T20:48:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
83022025-11-12T20:48:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
83032025-11-12T20:48:50.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
83042025-11-12T20:48:50.295ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.494114ms iteration = 14 status = "NoSitrep"
83052025-11-12T20:48:50.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83062025-11-12T20:48:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
83072025-11-12T20:48:50.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
83082025-11-12T20:48:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
83092025-11-12T20:48:50.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
83102025-11-12T20:48:50.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
83112025-11-12T20:48:50.303ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83122025-11-12T20:48:50.304ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
83132025-11-12T20:48:50.305ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
83142025-11-12T20:48:50.305ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.998361ms iteration = 14 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
83152025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
83162025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83172025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
83182025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 41 reason = Timeout
83192025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83202025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83212025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83222025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83232025-11-12T20:48:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83242025-11-12T20:48:55.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83252025-11-12T20:48:55.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83262025-11-12T20:48:55.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83272025-11-12T20:48:55.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83282025-11-12T20:48:55.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.918831ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
83292025-11-12T20:48:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83302025-11-12T20:48:55.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83312025-11-12T20:48:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83322025-11-12T20:48:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
83332025-11-12T20:48:55.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83342025-11-12T20:48:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83352025-11-12T20:48:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83362025-11-12T20:48:55.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
83372025-11-12T20:48:55.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83382025-11-12T20:48:55.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.996097ms iteration = 41 status = {"config_updated":false}
83392025-11-12T20:48:55.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83402025-11-12T20:48:55.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
83412025-11-12T20:48:55.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83422025-11-12T20:48:55.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
83432025-11-12T20:48:55.310ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
83442025-11-12T20:48:55.310ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.888963ms iteration = 21 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
83452025-11-12T20:48:57.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 309 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 68bb0715-cb87-4a45-90c9-f54c3d891d3d response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
83462025-11-12T20:49:00.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 42 reason = Timeout
83472025-11-12T20:49:00.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83482025-11-12T20:49:00.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83492025-11-12T20:49:00.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83502025-11-12T20:49:00.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
83512025-11-12T20:49:00.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
83522025-11-12T20:49:00.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
83532025-11-12T20:49:00.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
83542025-11-12T20:49:00.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.594446ms iteration = 42 status = {"config_updated":false}
83552025-11-12T20:49:02.917ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:58087
83562025-11-12T20:49:02.917ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = 4a9f7bbe-02d7-45ce-b19f-b1509745d13d uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
83572025-11-12T20:49:02.917ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = 4a9f7bbe-02d7-45ce-b19f-b1509745d13d roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
83582025-11-12T20:49:02.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = 4a9f7bbe-02d7-45ce-b19f-b1509745d13d resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
83592025-11-12T20:49:02.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2637 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = 4a9f7bbe-02d7-45ce-b19f-b1509745d13d response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
83602025-11-12T20:49:02.920ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = b8cce523-2907-4f1f-afcf-b9167a9dd83e uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
83612025-11-12T20:49:02.920ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = b8cce523-2907-4f1f-afcf-b9167a9dd83e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
83622025-11-12T20:49:02.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = b8cce523-2907-4f1f-afcf-b9167a9dd83e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
83632025-11-12T20:49:02.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2359 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58087 req_id = b8cce523-2907-4f1f-afcf-b9167a9dd83e response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
83642025-11-12T20:49:05.277ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
83652025-11-12T20:49:05.277ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 8 reason = Timeout
83662025-11-12T20:49:05.277ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
83672025-11-12T20:49:05.277ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
83682025-11-12T20:49:05.277ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
83692025-11-12T20:49:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
83702025-11-12T20:49:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
83712025-11-12T20:49:05.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 20 reason = Timeout
83722025-11-12T20:49:05.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
83732025-11-12T20:49:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
83742025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
83752025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
83762025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
83772025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
83782025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
83792025-11-12T20:49:05.280ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
83802025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
83812025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
83822025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
83832025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
83842025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
83852025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83862025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
83872025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
83882025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
83892025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
83902025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
83912025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
83922025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
83932025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83942025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
83952025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
83962025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 8 reason = Timeout
83972025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
83982025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
83992025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
84002025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84012025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
84022025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84032025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
84042025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84052025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
84062025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
84072025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84082025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84092025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 43 reason = Timeout
84102025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
84112025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
84122025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84132025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
84142025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
84152025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
84162025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
84172025-11-12T20:49:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
84182025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
84192025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
84202025-11-12T20:49:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1140, tv_nsec: 775318371 } })
84212025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
84222025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84232025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84242025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
84252025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84262025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
84272025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84282025-11-12T20:49:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1140, tv_nsec: 775500336 } })
84292025-11-12T20:49:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84302025-11-12T20:49:05.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.440983ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
84312025-11-12T20:49:05.282ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84322025-11-12T20:49:05.283ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84332025-11-12T20:49:05.283ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84342025-11-12T20:49:05.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84352025-11-12T20:49:05.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.47129ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
84362025-11-12T20:49:05.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84372025-11-12T20:49:05.284ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1140, tv_nsec: 778503884 } })
84382025-11-12T20:49:05.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84392025-11-12T20:49:05.284ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
84402025-11-12T20:49:05.284ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
84412025-11-12T20:49:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84422025-11-12T20:49:05.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 15 reason = Timeout
84432025-11-12T20:49:05.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
84442025-11-12T20:49:05.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84452025-11-12T20:49:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
84462025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
84472025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
84482025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
84492025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
84502025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
84512025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
84522025-11-12T20:49:05.288ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
84532025-11-12T20:49:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
84542025-11-12T20:49:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
84552025-11-12T20:49:05.289ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84562025-11-12T20:49:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
84572025-11-12T20:49:05.289ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84582025-11-12T20:49:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84592025-11-12T20:49:05.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84602025-11-12T20:49:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
84612025-11-12T20:49:05.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84622025-11-12T20:49:05.291ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84632025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
84642025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
84652025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
84662025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
84672025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
84682025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
84692025-11-12T20:49:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
84702025-11-12T20:49:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84712025-11-12T20:49:05.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84722025-11-12T20:49:05.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 15 reason = Timeout
84732025-11-12T20:49:05.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
84742025-11-12T20:49:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
84752025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
84762025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.014423ms iteration = 43 status = {"config_updated":false}
84772025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84782025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 16.812135ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
84792025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
84802025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
84812025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
84822025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
84832025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84842025-11-12T20:49:05.294ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84852025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
84862025-11-12T20:49:05.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
84872025-11-12T20:49:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
84882025-11-12T20:49:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
84892025-11-12T20:49:05.295ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1140, tv_nsec: 789412126 } })
84902025-11-12T20:49:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84912025-11-12T20:49:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
84922025-11-12T20:49:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
84932025-11-12T20:49:05.295ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
84942025-11-12T20:49:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
84952025-11-12T20:49:05.296ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
84962025-11-12T20:49:05.296ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
84972025-11-12T20:49:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
84982025-11-12T20:49:05.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
84992025-11-12T20:49:05.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
85002025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
85012025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
85022025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85032025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85042025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.239267ms iteration = 8 status = 0
85052025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
85062025-11-12T20:49:05.297ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
85072025-11-12T20:49:05.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
85082025-11-12T20:49:05.298ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85092025-11-12T20:49:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85102025-11-12T20:49:05.299ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49827f1a-3808-4b86-9622-c369fde9d0c0 original_time_started = 2025-11-12 20:48:36.725448 UTC
85112025-11-12T20:49:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 20.795818ms iteration = 20 status = {"Loaded":{"collection_id":"49827f1a-3808-4b86-9622-c369fde9d0c0","time_loaded":"2025-11-12T20:49:05.278963762Z","time_started":"2025-11-12T20:48:36.725448Z"}}
85122025-11-12T20:49:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 12.855402ms iteration = 15 status = "NoSitrep"
85132025-11-12T20:49:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
85142025-11-12T20:49:05.300ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
85152025-11-12T20:49:05.300ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 20.390028ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
85162025-11-12T20:49:05.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85172025-11-12T20:49:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
85182025-11-12T20:49:05.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85192025-11-12T20:49:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
85202025-11-12T20:49:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
85212025-11-12T20:49:05.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85222025-11-12T20:49:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85232025-11-12T20:49:05.306ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
85242025-11-12T20:49:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
85252025-11-12T20:49:05.308ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
85262025-11-12T20:49:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
85272025-11-12T20:49:05.312ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85282025-11-12T20:49:05.315ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
85292025-11-12T20:49:05.315ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
85302025-11-12T20:49:05.315ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 22.645686ms iteration = 15 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
85312025-11-12T20:49:05.324ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 44.178913ms iteration = 14 status = {"physical_disks_added":0}
85322025-11-12T20:49:05.329ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
85332025-11-12T20:49:05.329ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.182703ms iteration = 22 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
85342025-11-12T20:49:05.348ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
85352025-11-12T20:49:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
85362025-11-12T20:49:05.351ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
85372025-11-12T20:49:05.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
85382025-11-12T20:49:05.352ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85392025-11-12T20:49:05.352ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 72.390334ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
85402025-11-12T20:49:05.352ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 72.529957ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
85412025-11-12T20:49:05.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85422025-11-12T20:49:05.353ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 73.194004ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
85432025-11-12T20:49:05.354ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85442025-11-12T20:49:05.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
85452025-11-12T20:49:05.357ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85462025-11-12T20:49:05.357ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
85472025-11-12T20:49:05.357ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
85482025-11-12T20:49:05.357ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 76.91092ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
85492025-11-12T20:49:05.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85502025-11-12T20:49:05.358ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
85512025-11-12T20:49:05.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
85522025-11-12T20:49:05.358ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85532025-11-12T20:49:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
85542025-11-12T20:49:05.360ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 79.678242ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
85552025-11-12T20:49:05.388ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
85562025-11-12T20:49:05.388ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed43dd6b-2e9d-4277-b7e3-5afafa162914", "content-length": "78", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
85572025-11-12T20:49:05.388ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
85582025-11-12T20:49:05.388ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
85592025-11-12T20:49:05.388ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
85602025-11-12T20:49:05.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
85612025-11-12T20:49:05.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
85622025-11-12T20:49:05.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
85632025-11-12T20:49:05.390ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
85642025-11-12T20:49:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
85652025-11-12T20:49:05.391ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
85662025-11-12T20:49:05.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
85672025-11-12T20:49:05.394ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
85682025-11-12T20:49:05.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
85692025-11-12T20:49:05.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
85702025-11-12T20:49:05.394ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
85712025-11-12T20:49:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
85722025-11-12T20:49:05.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
85732025-11-12T20:49:05.395ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
85742025-11-12T20:49:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
85752025-11-12T20:49:05.397ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
85762025-11-12T20:49:05.397ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
85772025-11-12T20:49:05.397ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
85782025-11-12T20:49:05.397ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae17ef8e-f9c8-410c-b869-b5ee45632851", "content-length": "81", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
85792025-11-12T20:49:05.398ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 120.509549ms iteration = 8 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
85802025-11-12T20:49:05.398ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
85812025-11-12T20:49:05.398ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
85822025-11-12T20:49:05.398ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "91ef1f4c-d003-4cb6-abd9-ec818a390a39", "content-length": "26", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
85832025-11-12T20:49:05.398ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
85842025-11-12T20:49:05.399ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
85852025-11-12T20:49:05.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
85862025-11-12T20:49:05.400ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
85872025-11-12T20:49:05.400ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
85882025-11-12T20:49:05.400ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "36182e79-114a-48d8-8413-4c62d031cfec", "content-length": "26", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
85892025-11-12T20:49:05.400ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
85902025-11-12T20:49:05.400ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
85912025-11-12T20:49:05.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 121.131515ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
85922025-11-12T20:49:05.402ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
85932025-11-12T20:49:05.409ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
85942025-11-12T20:49:05.409ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 128.858845ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
85952025-11-12T20:49:05.416ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85962025-11-12T20:49:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
85972025-11-12T20:49:05.418ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85982025-11-12T20:49:05.420ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
85992025-11-12T20:49:05.420ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86002025-11-12T20:49:05.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
86012025-11-12T20:49:05.421ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86022025-11-12T20:49:05.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
86032025-11-12T20:49:05.423ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86042025-11-12T20:49:05.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
86052025-11-12T20:49:05.440ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
86062025-11-12T20:49:05.440ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
86072025-11-12T20:49:05.440ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "a42788b7-2297-42e4-9419-a04734e659eb", "content-length": "81", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86082025-11-12T20:49:05.443ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
86092025-11-12T20:49:05.443ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
86102025-11-12T20:49:05.444ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aafd539-52fc-4e44-afbb-a8b3e585f0ae", "content-length": "81", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86112025-11-12T20:49:05.446ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 165.660277ms iteration = 8 status = {"disabled":false,"errors":[],"sps":[]}
86122025-11-12T20:49:05.495ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 218.046032ms iteration = 8 status = {}
86132025-11-12T20:49:05.510ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 5064607 } })
86142025-11-12T20:49:05.519ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
86152025-11-12T20:49:05.519ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
86162025-11-12T20:49:05.519ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "23adda6f-c04b-416c-a469-2f0d40473494", "content-length": "372", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86172025-11-12T20:49:05.519ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86182025-11-12T20:49:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86192025-11-12T20:49:05.525ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
86202025-11-12T20:49:05.527ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3139a910-d144-41e1-85a4-f289ca6ea176", "content-length": "691", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86212025-11-12T20:49:05.527ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
86222025-11-12T20:49:05.527ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
86232025-11-12T20:49:05.527ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
86242025-11-12T20:49:05.529ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 248.521621ms iteration = 8 status = {}
86252025-11-12T20:49:05.574ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
86262025-11-12T20:49:05.574ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
86272025-11-12T20:49:05.574ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9075a590-d094-4094-affd-5a081966b84a", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86282025-11-12T20:49:05.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86292025-11-12T20:49:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86302025-11-12T20:49:05.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86312025-11-12T20:49:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86322025-11-12T20:49:05.581ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86332025-11-12T20:49:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86342025-11-12T20:49:05.584ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
86352025-11-12T20:49:05.584ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
86362025-11-12T20:49:05.585ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "76ba00e6-28f4-4227-a459-8882f93ec799", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86372025-11-12T20:49:05.587ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 307.044956ms iteration = 12 status = {}
86382025-11-12T20:49:05.589ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86392025-11-12T20:49:05.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86402025-11-12T20:49:05.592ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
86412025-11-12T20:49:05.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86422025-11-12T20:49:05.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86432025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
86442025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
86452025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
86462025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
86472025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
86482025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
86492025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
86502025-11-12T20:49:05.602ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
86512025-11-12T20:49:05.603ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
86522025-11-12T20:49:05.603ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
86532025-11-12T20:49:05.603ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
86542025-11-12T20:49:05.603ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86552025-11-12T20:49:05.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86562025-11-12T20:49:05.605ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
86572025-11-12T20:49:05.606ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
86582025-11-12T20:49:05.716ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "dbd8ef9b-46d4-4478-ac0b-5801818b3b73", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86592025-11-12T20:49:05.716ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
86602025-11-12T20:49:05.716ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
86612025-11-12T20:49:05.716ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fb2bc2f-761c-43c7-9225-932cc381078d", "content-length": "682", "date": "Wed, 12 Nov 2025 20:49:05 GMT"} })
86622025-11-12T20:49:05.716ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86632025-11-12T20:49:05.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86642025-11-12T20:49:05.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86652025-11-12T20:49:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86662025-11-12T20:49:05.723ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86672025-11-12T20:49:05.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86682025-11-12T20:49:05.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
86692025-11-12T20:49:05.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
86702025-11-12T20:49:05.733ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 453.433414ms iteration = 8 status = {}
86712025-11-12T20:49:05.979ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
86722025-11-12T20:49:05.979ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
86732025-11-12T20:49:06.009ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
86742025-11-12T20:49:06.009ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:55157
86752025-11-12T20:49:06.010ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:55157 req_id = 40036ac5-dc33-4f01-9c6c-2b2a47463ce3 uri = /metrics/producers version = 1.0.0
86762025-11-12T20:49:06.010ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:55157 req_id = 40036ac5-dc33-4f01-9c6c-2b2a47463ce3 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
86772025-11-12T20:49:06.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:55157 req_id = 40036ac5-dc33-4f01-9c6c-2b2a47463ce3 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
86782025-11-12T20:49:06.055ZINFOnexus: assigned collector to new producer collector_id = ecb0f932-e2e8-41d3-85a9-494e5cae5bce file = nexus/src/app/oximeter.rs:76 producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
86792025-11-12T20:49:06.055ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 45156 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:55157 req_id = 40036ac5-dc33-4f01-9c6c-2b2a47463ce3 response_code = 201 uri = /metrics/producers
86802025-11-12T20:49:06.055ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "40036ac5-dc33-4f01-9c6c-2b2a47463ce3", "content-length": "41", "date": "Wed, 12 Nov 2025 20:49:06 GMT"} })
86812025-11-12T20:49:06.056ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
86822025-11-12T20:49:07.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 370 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 98a53e87-8da6-43f7-9934-6953096e0db7 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
86832025-11-12T20:49:10.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 44 reason = Timeout
86842025-11-12T20:49:10.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
86852025-11-12T20:49:10.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
86862025-11-12T20:49:10.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86872025-11-12T20:49:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
86882025-11-12T20:49:10.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
86892025-11-12T20:49:10.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
86902025-11-12T20:49:10.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
86912025-11-12T20:49:10.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.251602ms iteration = 44 status = {"config_updated":false}
86922025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
86932025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
86942025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86952025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86962025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 45 reason = Timeout
86972025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
86982025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
86992025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87002025-11-12T20:49:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87012025-11-12T20:49:15.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87022025-11-12T20:49:15.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87032025-11-12T20:49:15.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87042025-11-12T20:49:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87052025-11-12T20:49:15.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.85826ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
87062025-11-12T20:49:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
87072025-11-12T20:49:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87082025-11-12T20:49:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87092025-11-12T20:49:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
87102025-11-12T20:49:15.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87112025-11-12T20:49:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87122025-11-12T20:49:15.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87132025-11-12T20:49:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87142025-11-12T20:49:15.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
87152025-11-12T20:49:15.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.378667ms iteration = 45 status = {"config_updated":false}
87162025-11-12T20:49:15.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87172025-11-12T20:49:15.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
87182025-11-12T20:49:15.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87192025-11-12T20:49:15.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87202025-11-12T20:49:15.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
87212025-11-12T20:49:15.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.741984ms iteration = 23 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
87222025-11-12T20:49:17.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 415 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = f428329e-b70b-46f2-8106-78a154bdbb53 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
87232025-11-12T20:49:20.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 21 reason = Timeout
87242025-11-12T20:49:20.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
87252025-11-12T20:49:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
87262025-11-12T20:49:20.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 46 reason = Timeout
87272025-11-12T20:49:20.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87282025-11-12T20:49:20.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87292025-11-12T20:49:20.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87302025-11-12T20:49:20.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87312025-11-12T20:49:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
87322025-11-12T20:49:20.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
87332025-11-12T20:49:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
87342025-11-12T20:49:20.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87352025-11-12T20:49:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
87362025-11-12T20:49:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87372025-11-12T20:49:20.286ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49827f1a-3808-4b86-9622-c369fde9d0c0 original_time_started = 2025-11-12 20:48:36.725448 UTC
87382025-11-12T20:49:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.440603ms iteration = 21 status = {"Loaded":{"collection_id":"49827f1a-3808-4b86-9622-c369fde9d0c0","time_loaded":"2025-11-12T20:49:20.279467076Z","time_started":"2025-11-12T20:48:36.725448Z"}}
87392025-11-12T20:49:20.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 16 reason = Timeout
87402025-11-12T20:49:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
87412025-11-12T20:49:20.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
87422025-11-12T20:49:20.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.561506ms iteration = 46 status = {"config_updated":false}
87432025-11-12T20:49:20.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
87442025-11-12T20:49:20.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87452025-11-12T20:49:20.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
87462025-11-12T20:49:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
87472025-11-12T20:49:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
87482025-11-12T20:49:20.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 16 reason = Timeout
87492025-11-12T20:49:20.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
87502025-11-12T20:49:20.293ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.358225ms iteration = 16 status = "NoSitrep"
87512025-11-12T20:49:20.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
87522025-11-12T20:49:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
87532025-11-12T20:49:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
87542025-11-12T20:49:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87552025-11-12T20:49:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
87562025-11-12T20:49:20.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
87572025-11-12T20:49:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
87582025-11-12T20:49:20.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
87592025-11-12T20:49:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
87602025-11-12T20:49:20.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87612025-11-12T20:49:20.303ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
87622025-11-12T20:49:20.303ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
87632025-11-12T20:49:20.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.400829ms iteration = 16 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
87642025-11-12T20:49:25.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
87652025-11-12T20:49:25.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
87662025-11-12T20:49:25.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87672025-11-12T20:49:25.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 47 reason = Timeout
87682025-11-12T20:49:25.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87692025-11-12T20:49:25.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87702025-11-12T20:49:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87712025-11-12T20:49:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87722025-11-12T20:49:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87732025-11-12T20:49:25.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87742025-11-12T20:49:25.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87752025-11-12T20:49:25.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87762025-11-12T20:49:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87772025-11-12T20:49:25.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.715126ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
87782025-11-12T20:49:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
87792025-11-12T20:49:25.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87802025-11-12T20:49:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87812025-11-12T20:49:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
87822025-11-12T20:49:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87832025-11-12T20:49:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87842025-11-12T20:49:25.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
87852025-11-12T20:49:25.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.169695ms iteration = 47 status = {"config_updated":false}
87862025-11-12T20:49:25.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87872025-11-12T20:49:25.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87882025-11-12T20:49:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87892025-11-12T20:49:25.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
87902025-11-12T20:49:25.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87912025-11-12T20:49:25.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87922025-11-12T20:49:25.307ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
87932025-11-12T20:49:25.307ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.883436ms iteration = 24 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
87942025-11-12T20:49:27.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 378 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 5f4e7553-3ded-4308-8565-79c93bdece1f response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
87952025-11-12T20:49:30.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 48 reason = Timeout
87962025-11-12T20:49:30.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87972025-11-12T20:49:30.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87982025-11-12T20:49:30.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87992025-11-12T20:49:30.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
88002025-11-12T20:49:30.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88012025-11-12T20:49:30.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
88022025-11-12T20:49:30.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
88032025-11-12T20:49:30.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.098109ms iteration = 48 status = {"config_updated":false}
88042025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
88052025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
88062025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
88072025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
88082025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
88092025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
88102025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
88112025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
88122025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
88132025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
88142025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
88152025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 9 reason = Timeout
88162025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
88172025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
88182025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
88192025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
88202025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
88212025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
88222025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
88232025-11-12T20:49:35.277ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
88242025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
88252025-11-12T20:49:35.277ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = ExternalNetworking step id = Ensure
88262025-11-12T20:49:35.277ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:39:35.277904642 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
88272025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
88282025-11-12T20:49:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88292025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
88302025-11-12T20:49:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1170, tv_nsec: 772650958 } })
88312025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
88322025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
88332025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
88342025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88352025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
88362025-11-12T20:49:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1170, tv_nsec: 772846474 } })
88372025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
88382025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 22 reason = Timeout
88392025-11-12T20:49:35.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
88402025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
88412025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.338625ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
88422025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
88432025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
88442025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.283173ms iteration = 5 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
88452025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
88462025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
88472025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
88482025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
88492025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 9 reason = Timeout
88502025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
88512025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Timeout
88522025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
88532025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
88542025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
88552025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88562025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88572025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
88582025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
88592025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
88602025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
88612025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
88622025-11-12T20:49:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
88632025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
88642025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
88652025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
88662025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88672025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
88682025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88692025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
88702025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
88712025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
88722025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
88732025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
88742025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
88752025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
88762025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
88772025-11-12T20:49:35.280ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
88782025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88792025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 49 reason = Timeout
88802025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88812025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
88822025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
88832025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
88842025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
88852025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
88862025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
88872025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
88882025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
88892025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
88902025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
88912025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
88922025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
88932025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
88942025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
88952025-11-12T20:49:35.280ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
88962025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
88972025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
88982025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
88992025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
89002025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
89012025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
89022025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
89032025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
89042025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89052025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
89062025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
89072025-11-12T20:49:35.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1170, tv_nsec: 774903107 } })
89082025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89092025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
89102025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89112025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89122025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89132025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89142025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
89152025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
89162025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
89172025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
89182025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
89192025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
89202025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
89212025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
89222025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
89232025-11-12T20:49:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
89242025-11-12T20:49:35.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89252025-11-12T20:49:35.286ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:39:35.277904642 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
89262025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.183752ms iteration = 5 status = {"expiration":"2025-11-12T20:39:35.277904642Z","pruned":[]}
89272025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
89282025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 6.216181ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
89292025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.330174ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
89302025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89312025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89322025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
89332025-11-12T20:49:35.286ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1170, tv_nsec: 781028865 } })
89342025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 17 reason = Timeout
89352025-11-12T20:49:35.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
89362025-11-12T20:49:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
89372025-11-12T20:49:35.287ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
89382025-11-12T20:49:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
89392025-11-12T20:49:35.287ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
89402025-11-12T20:49:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
89412025-11-12T20:49:35.288ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1170, tv_nsec: 782517954 } })
89422025-11-12T20:49:35.289ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89432025-11-12T20:49:35.290ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89442025-11-12T20:49:35.290ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89452025-11-12T20:49:35.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89462025-11-12T20:49:35.291ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89472025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89482025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 5 reason = Timeout
89492025-11-12T20:49:35.293ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
89502025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 17 reason = Timeout
89512025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
89522025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
89532025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89542025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
89552025-11-12T20:49:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
89562025-11-12T20:49:35.294ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
89572025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 14.773942ms iteration = 5 status = {"by_rx":{},"error":null}
89582025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.486302ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
89592025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89602025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89612025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
89622025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89632025-11-12T20:49:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
89642025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
89652025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
89662025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 16.719983ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
89672025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
89682025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89692025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
89702025-11-12T20:49:35.296ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
89712025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
89722025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
89732025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
89742025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89752025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
89762025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
89772025-11-12T20:49:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
89782025-11-12T20:49:35.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89792025-11-12T20:49:35.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89802025-11-12T20:49:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89812025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
89822025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89832025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
89842025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
89852025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89862025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
89872025-11-12T20:49:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
89882025-11-12T20:49:35.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89892025-11-12T20:49:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89902025-11-12T20:49:35.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
89912025-11-12T20:49:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
89922025-11-12T20:49:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
89932025-11-12T20:49:35.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89942025-11-12T20:49:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
89952025-11-12T20:49:35.305ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
89962025-11-12T20:49:35.305ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
89972025-11-12T20:49:35.305ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
89982025-11-12T20:49:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
89992025-11-12T20:49:35.306ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90002025-11-12T20:49:35.306ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90012025-11-12T20:49:35.306ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90022025-11-12T20:49:35.309ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49827f1a-3808-4b86-9622-c369fde9d0c0 original_time_started = 2025-11-12 20:48:36.725448 UTC
90032025-11-12T20:49:35.309ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 30.550771ms iteration = 22 status = {"Loaded":{"collection_id":"49827f1a-3808-4b86-9622-c369fde9d0c0","time_loaded":"2025-11-12T20:49:35.278706592Z","time_started":"2025-11-12T20:48:36.725448Z"}}
90042025-11-12T20:49:35.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
90052025-11-12T20:49:35.310ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
90062025-11-12T20:49:35.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
90072025-11-12T20:49:35.310ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
90082025-11-12T20:49:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
90092025-11-12T20:49:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
90102025-11-12T20:49:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
90112025-11-12T20:49:35.312ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90122025-11-12T20:49:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
90132025-11-12T20:49:35.312ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90142025-11-12T20:49:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90152025-11-12T20:49:35.313ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
90162025-11-12T20:49:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
90172025-11-12T20:49:35.314ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
90182025-11-12T20:49:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
90192025-11-12T20:49:35.315ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
90202025-11-12T20:49:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
90212025-11-12T20:49:35.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
90222025-11-12T20:49:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90232025-11-12T20:49:35.317ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1170, tv_nsec: 811902074 } })
90242025-11-12T20:49:35.318ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
90252025-11-12T20:49:35.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90262025-11-12T20:49:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90272025-11-12T20:49:35.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.751199ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
90282025-11-12T20:49:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
90292025-11-12T20:49:35.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90302025-11-12T20:49:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
90312025-11-12T20:49:35.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
90322025-11-12T20:49:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
90332025-11-12T20:49:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
90342025-11-12T20:49:35.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90352025-11-12T20:49:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
90362025-11-12T20:49:35.334ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 47.335815ms iteration = 17 status = "NoSitrep"
90372025-11-12T20:49:35.335ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
90382025-11-12T20:49:35.336ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90392025-11-12T20:49:35.337ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
90402025-11-12T20:49:35.337ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90412025-11-12T20:49:35.339ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90422025-11-12T20:49:35.339ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
90432025-11-12T20:49:35.339ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
90442025-11-12T20:49:35.340ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
90452025-11-12T20:49:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
90462025-11-12T20:49:35.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90472025-11-12T20:49:35.341ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90482025-11-12T20:49:35.341ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90492025-11-12T20:49:35.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
90502025-11-12T20:49:35.342ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
90512025-11-12T20:49:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90522025-11-12T20:49:35.342ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 62.296892ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
90532025-11-12T20:49:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
90542025-11-12T20:49:35.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
90552025-11-12T20:49:35.344ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
90562025-11-12T20:49:35.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
90572025-11-12T20:49:35.344ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 64.615092ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
90582025-11-12T20:49:35.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90592025-11-12T20:49:35.346ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
90602025-11-12T20:49:35.347ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
90612025-11-12T20:49:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
90622025-11-12T20:49:35.348ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
90632025-11-12T20:49:35.348ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
90642025-11-12T20:49:35.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
90652025-11-12T20:49:35.349ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
90662025-11-12T20:49:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
90672025-11-12T20:49:35.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90682025-11-12T20:49:35.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
90692025-11-12T20:49:35.351ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
90702025-11-12T20:49:35.353ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
90712025-11-12T20:49:35.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
90722025-11-12T20:49:35.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
90732025-11-12T20:49:35.354ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
90742025-11-12T20:49:35.354ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 74.59109ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
90752025-11-12T20:49:35.358ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
90762025-11-12T20:49:35.359ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
90772025-11-12T20:49:35.366ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
90782025-11-12T20:49:35.367ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
90792025-11-12T20:49:35.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
90802025-11-12T20:49:35.369ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90812025-11-12T20:49:35.370ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
90822025-11-12T20:49:35.370ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = SledList step id = Fetch
90832025-11-12T20:49:35.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90842025-11-12T20:49:35.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90852025-11-12T20:49:35.371ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
90862025-11-12T20:49:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
90872025-11-12T20:49:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
90882025-11-12T20:49:35.373ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 93.543441ms iteration = 9 status = 0
90892025-11-12T20:49:35.373ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
90902025-11-12T20:49:35.373ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 80.225846ms iteration = 17 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
90912025-11-12T20:49:35.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90922025-11-12T20:49:35.375ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90932025-11-12T20:49:35.378ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
90942025-11-12T20:49:35.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
90952025-11-12T20:49:35.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
90962025-11-12T20:49:35.380ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
90972025-11-12T20:49:35.380ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
90982025-11-12T20:49:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
90992025-11-12T20:49:35.380ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91002025-11-12T20:49:35.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91012025-11-12T20:49:35.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91022025-11-12T20:49:35.382ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91032025-11-12T20:49:35.385ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
91042025-11-12T20:49:35.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91052025-11-12T20:49:35.385ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
91062025-11-12T20:49:35.385ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91072025-11-12T20:49:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91082025-11-12T20:49:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91092025-11-12T20:49:35.386ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
91102025-11-12T20:49:35.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91112025-11-12T20:49:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91122025-11-12T20:49:35.387ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
91132025-11-12T20:49:35.387ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
91142025-11-12T20:49:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
91152025-11-12T20:49:35.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91162025-11-12T20:49:35.391ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91172025-11-12T20:49:35.393ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91182025-11-12T20:49:35.395ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
91192025-11-12T20:49:35.395ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
91202025-11-12T20:49:35.395ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 115.119359ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
91212025-11-12T20:49:35.395ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91222025-11-12T20:49:35.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
91232025-11-12T20:49:35.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
91242025-11-12T20:49:35.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
91252025-11-12T20:49:35.397ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
91262025-11-12T20:49:35.397ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91272025-11-12T20:49:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91282025-11-12T20:49:35.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91292025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 120.17763ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
91302025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91312025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
91322025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91332025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
91342025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
91352025-11-12T20:49:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91362025-11-12T20:49:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
91372025-11-12T20:49:35.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91382025-11-12T20:49:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91392025-11-12T20:49:35.408ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
91402025-11-12T20:49:35.408ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 128.205408ms iteration = 49 status = {"config_updated":false}
91412025-11-12T20:49:35.408ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
91422025-11-12T20:49:35.408ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 128.490665ms iteration = 25 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
91432025-11-12T20:49:35.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91442025-11-12T20:49:35.412ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
91452025-11-12T20:49:35.412ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 132.51244ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
91462025-11-12T20:49:35.415ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 135.458226ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
91472025-11-12T20:49:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
91482025-11-12T20:49:35.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91492025-11-12T20:49:35.416ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91502025-11-12T20:49:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91512025-11-12T20:49:35.418ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
91522025-11-12T20:49:35.418ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
91532025-11-12T20:49:35.418ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
91542025-11-12T20:49:35.418ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
91552025-11-12T20:49:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
91562025-11-12T20:49:35.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91572025-11-12T20:49:35.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91582025-11-12T20:49:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
91592025-11-12T20:49:35.423ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91602025-11-12T20:49:35.426ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
91612025-11-12T20:49:35.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91622025-11-12T20:49:35.426ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
91632025-11-12T20:49:35.426ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
91642025-11-12T20:49:35.426ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
91652025-11-12T20:49:35.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
91662025-11-12T20:49:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
91672025-11-12T20:49:35.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
91682025-11-12T20:49:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
91692025-11-12T20:49:35.430ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
91702025-11-12T20:49:35.430ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
91712025-11-12T20:49:35.430ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
91722025-11-12T20:49:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
91732025-11-12T20:49:35.431ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
91742025-11-12T20:49:35.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
91752025-11-12T20:49:35.431ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91762025-11-12T20:49:35.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
91772025-11-12T20:49:35.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91782025-11-12T20:49:35.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91792025-11-12T20:49:35.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91802025-11-12T20:49:35.435ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 155.272898ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
91812025-11-12T20:49:35.437ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
91822025-11-12T20:49:35.437ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91832025-11-12T20:49:35.437ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "544555cc-b159-4083-b45d-90735bf96bff", "content-length": "26", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
91842025-11-12T20:49:35.437ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
91852025-11-12T20:49:35.437ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
91862025-11-12T20:49:35.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
91872025-11-12T20:49:35.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
91882025-11-12T20:49:35.439ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
91892025-11-12T20:49:35.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91902025-11-12T20:49:35.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91912025-11-12T20:49:35.441ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
91922025-11-12T20:49:35.441ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
91932025-11-12T20:49:35.441ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
91942025-11-12T20:49:35.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
91952025-11-12T20:49:35.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
91962025-11-12T20:49:35.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
91972025-11-12T20:49:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
91982025-11-12T20:49:35.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91992025-11-12T20:49:35.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92002025-11-12T20:49:35.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
92012025-11-12T20:49:35.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92022025-11-12T20:49:35.450ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
92032025-11-12T20:49:35.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92042025-11-12T20:49:35.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92052025-11-12T20:49:35.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92062025-11-12T20:49:35.451ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
92072025-11-12T20:49:35.451ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
92082025-11-12T20:49:35.451ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 171.611891ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
92092025-11-12T20:49:35.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92102025-11-12T20:49:35.455ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 174.994899ms iteration = 15 status = {"physical_disks_added":0}
92112025-11-12T20:49:35.455ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
92122025-11-12T20:49:35.455ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
92132025-11-12T20:49:35.455ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
92142025-11-12T20:49:35.455ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 177.446972ms iteration = 5 status = {"generation":1}
92152025-11-12T20:49:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92162025-11-12T20:49:35.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92172025-11-12T20:49:35.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
92182025-11-12T20:49:35.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92192025-11-12T20:49:35.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92202025-11-12T20:49:35.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92212025-11-12T20:49:35.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92222025-11-12T20:49:35.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92232025-11-12T20:49:35.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
92242025-11-12T20:49:35.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92252025-11-12T20:49:35.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92262025-11-12T20:49:35.491ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
92272025-11-12T20:49:35.491ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 213.899186ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
92282025-11-12T20:49:35.500ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
92292025-11-12T20:49:35.500ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd6240b0-69ee-4257-86d4-d572b9b957aa", "content-length": "78", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
92302025-11-12T20:49:35.500ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
92312025-11-12T20:49:35.500ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
92322025-11-12T20:49:35.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
92332025-11-12T20:49:35.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
92342025-11-12T20:49:35.501ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
92352025-11-12T20:49:35.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
92362025-11-12T20:49:35.503ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
92372025-11-12T20:49:35.504ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
92382025-11-12T20:49:35.504ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
92392025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
92402025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
92412025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
92422025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
92432025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
92442025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
92452025-11-12T20:49:35.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92462025-11-12T20:49:35.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
92472025-11-12T20:49:35.508ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
92482025-11-12T20:49:35.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
92492025-11-12T20:49:35.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
92502025-11-12T20:49:35.510ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
92512025-11-12T20:49:35.510ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
92522025-11-12T20:49:35.510ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
92532025-11-12T20:49:35.510ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 230.445194ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
92542025-11-12T20:49:35.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
92552025-11-12T20:49:35.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92562025-11-12T20:49:35.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92572025-11-12T20:49:35.512ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92582025-11-12T20:49:35.512ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
92592025-11-12T20:49:35.514ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
92602025-11-12T20:49:35.514ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
92612025-11-12T20:49:35.514ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "add85653-cbd6-45da-8783-4d4c3daea0f7", "content-length": "81", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
92622025-11-12T20:49:35.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92632025-11-12T20:49:35.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92642025-11-12T20:49:35.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92652025-11-12T20:49:35.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
92662025-11-12T20:49:35.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
92672025-11-12T20:49:35.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92682025-11-12T20:49:35.520ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
92692025-11-12T20:49:35.520ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
92702025-11-12T20:49:35.521ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
92712025-11-12T20:49:35.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 243.258476ms iteration = 5 status = {"generation":2}
92722025-11-12T20:49:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
92732025-11-12T20:49:35.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92742025-11-12T20:49:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92752025-11-12T20:49:35.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92762025-11-12T20:49:35.529ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
92772025-11-12T20:49:35.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92782025-11-12T20:49:35.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92792025-11-12T20:49:35.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
92802025-11-12T20:49:35.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
92812025-11-12T20:49:35.531ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
92822025-11-12T20:49:35.533ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 255.969815ms iteration = 9 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
92832025-11-12T20:49:35.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92842025-11-12T20:49:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92852025-11-12T20:49:35.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92862025-11-12T20:49:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92872025-11-12T20:49:35.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92882025-11-12T20:49:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
92892025-11-12T20:49:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92902025-11-12T20:49:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92912025-11-12T20:49:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92922025-11-12T20:49:35.553ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
92932025-11-12T20:49:35.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92942025-11-12T20:49:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92952025-11-12T20:49:35.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92962025-11-12T20:49:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92972025-11-12T20:49:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92982025-11-12T20:49:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92992025-11-12T20:49:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93002025-11-12T20:49:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
93012025-11-12T20:49:35.576ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = DeployNexusRecords step id = Ensure
93022025-11-12T20:49:35.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
93032025-11-12T20:49:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
93042025-11-12T20:49:35.580ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
93052025-11-12T20:49:35.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93062025-11-12T20:49:35.580ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
93072025-11-12T20:49:35.581ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
93082025-11-12T20:49:35.581ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5e8ae4c-8f91-4081-9811-f500592a5ca0", "content-length": "81", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
93092025-11-12T20:49:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
93102025-11-12T20:49:35.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
93112025-11-12T20:49:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
93122025-11-12T20:49:35.586ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93132025-11-12T20:49:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
93142025-11-12T20:49:35.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
93152025-11-12T20:49:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
93162025-11-12T20:49:35.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93172025-11-12T20:49:35.590ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
93182025-11-12T20:49:35.590ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
93192025-11-12T20:49:35.590ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "18621dab-f0eb-4329-9c96-4749f42b109b", "content-length": "81", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
93202025-11-12T20:49:35.592ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 312.737234ms iteration = 9 status = {"disabled":false,"errors":[],"sps":[]}
93212025-11-12T20:49:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
93222025-11-12T20:49:35.600ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93232025-11-12T20:49:35.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
93242025-11-12T20:49:35.602ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93252025-11-12T20:49:35.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
93262025-11-12T20:49:35.607ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93272025-11-12T20:49:35.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
93282025-11-12T20:49:35.609ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93292025-11-12T20:49:35.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
93302025-11-12T20:49:35.613ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93312025-11-12T20:49:35.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
93322025-11-12T20:49:35.623ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
93332025-11-12T20:49:35.623ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93342025-11-12T20:49:35.623ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "682e1f58-ac7b-4ddf-9912-5ab51edcd8fa", "content-length": "26", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
93352025-11-12T20:49:35.623ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93362025-11-12T20:49:35.626ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
93372025-11-12T20:49:35.629ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 352.079101ms iteration = 9 status = {}
93382025-11-12T20:49:35.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93392025-11-12T20:49:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93402025-11-12T20:49:35.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93412025-11-12T20:49:35.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93422025-11-12T20:49:35.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93432025-11-12T20:49:35.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
93442025-11-12T20:49:35.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = SledAgent step id = Ensure
93452025-11-12T20:49:35.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93462025-11-12T20:49:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93472025-11-12T20:49:35.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93482025-11-12T20:49:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93492025-11-12T20:49:35.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93502025-11-12T20:49:35.663ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93512025-11-12T20:49:35.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93522025-11-12T20:49:35.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93532025-11-12T20:49:35.665ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
93542025-11-12T20:49:35.666ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "c393a231-08ca-4b49-b8c2-11294a24ca4c", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
93552025-11-12T20:49:35.666ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
93562025-11-12T20:49:35.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93572025-11-12T20:49:35.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93582025-11-12T20:49:35.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93592025-11-12T20:49:35.670ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
93602025-11-12T20:49:35.670ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 377.274293ms iteration = 5 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
93612025-11-12T20:49:35.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
93622025-11-12T20:49:35.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93632025-11-12T20:49:35.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93642025-11-12T20:49:35.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93652025-11-12T20:49:35.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93662025-11-12T20:49:35.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93672025-11-12T20:49:35.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93682025-11-12T20:49:35.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93692025-11-12T20:49:35.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93702025-11-12T20:49:35.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93712025-11-12T20:49:35.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93722025-11-12T20:49:35.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93732025-11-12T20:49:35.693ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
93742025-11-12T20:49:35.694ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "1a7a6964-753e-4dbf-b836-264d1bc085e6", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
93752025-11-12T20:49:35.694ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
93762025-11-12T20:49:35.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93772025-11-12T20:49:35.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93782025-11-12T20:49:35.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93792025-11-12T20:49:35.696ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "785189a5-e9fd-490a-b249-8e0bd30069e2", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
93802025-11-12T20:49:35.696ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
93812025-11-12T20:49:35.699ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
93822025-11-12T20:49:35.699ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 421.471267ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
93832025-11-12T20:49:35.729ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
93842025-11-12T20:49:35.729ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
93852025-11-12T20:49:35.729ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "406252d0-67aa-48db-87f0-936aa541b707", "content-length": "372", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
93862025-11-12T20:49:35.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
93872025-11-12T20:49:35.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
93882025-11-12T20:49:35.730ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
93892025-11-12T20:49:35.730ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 450.374465ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
93902025-11-12T20:49:35.738ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
93912025-11-12T20:49:35.744ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
93922025-11-12T20:49:35.744ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
93932025-11-12T20:49:35.744ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a75dac9c-953d-465a-befa-4f7ed7138869 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
93942025-11-12T20:49:35.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93952025-11-12T20:49:35.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93962025-11-12T20:49:35.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93972025-11-12T20:49:35.748ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 242489297 } })
93982025-11-12T20:49:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93992025-11-12T20:49:35.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94002025-11-12T20:49:35.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94012025-11-12T20:49:35.757ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
94022025-11-12T20:49:35.757ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
94032025-11-12T20:49:35.757ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 477.458466ms iteration = 13 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
94042025-11-12T20:49:35.785ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
94052025-11-12T20:49:35.787ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "5798691f-be25-4a36-b6a6-db9115564d79", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
94062025-11-12T20:49:35.789ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = FirewallRules step id = Ensure
94072025-11-12T20:49:35.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94082025-11-12T20:49:35.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94092025-11-12T20:49:35.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94102025-11-12T20:49:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94112025-11-12T20:49:35.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94122025-11-12T20:49:35.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94132025-11-12T20:49:35.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94142025-11-12T20:49:35.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94152025-11-12T20:49:35.799ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
94162025-11-12T20:49:35.799ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
94172025-11-12T20:49:35.800ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "463aa42f-b6ef-4bdf-9594-0960234d9468", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
94182025-11-12T20:49:35.800ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94192025-11-12T20:49:35.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94202025-11-12T20:49:35.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94212025-11-12T20:49:35.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94222025-11-12T20:49:35.802ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94232025-11-12T20:49:35.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94242025-11-12T20:49:35.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94252025-11-12T20:49:35.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94262025-11-12T20:49:35.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94272025-11-12T20:49:35.806ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94282025-11-12T20:49:35.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94292025-11-12T20:49:35.809ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
94302025-11-12T20:49:35.809ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
94312025-11-12T20:49:35.810ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f6b7f456-757d-4bb7-b382-e6f61faaa716", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
94322025-11-12T20:49:35.811ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
94332025-11-12T20:49:35.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94342025-11-12T20:49:35.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94352025-11-12T20:49:35.811ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 531.774031ms iteration = 13 status = {}
94362025-11-12T20:49:35.811ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Dependency
94372025-11-12T20:49:35.811ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
94382025-11-12T20:49:35.811ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94392025-11-12T20:49:35.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94402025-11-12T20:49:35.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94412025-11-12T20:49:35.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94422025-11-12T20:49:35.814ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94432025-11-12T20:49:35.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94442025-11-12T20:49:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94452025-11-12T20:49:35.819ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
94462025-11-12T20:49:35.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94472025-11-12T20:49:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94482025-11-12T20:49:35.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94492025-11-12T20:49:35.820ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94502025-11-12T20:49:35.824ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
94512025-11-12T20:49:35.824ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94522025-11-12T20:49:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
94532025-11-12T20:49:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94542025-11-12T20:49:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94552025-11-12T20:49:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94562025-11-12T20:49:35.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94572025-11-12T20:49:35.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94582025-11-12T20:49:35.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94592025-11-12T20:49:35.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94602025-11-12T20:49:35.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94612025-11-12T20:49:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94622025-11-12T20:49:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94632025-11-12T20:49:35.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94642025-11-12T20:49:35.835ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
94652025-11-12T20:49:35.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94662025-11-12T20:49:35.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94672025-11-12T20:49:35.838ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "77967dba-6a7f-4619-a9ea-d97d4a29b173", "content-length": "691", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
94682025-11-12T20:49:35.838ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
94692025-11-12T20:49:35.838ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
94702025-11-12T20:49:35.838ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
94712025-11-12T20:49:35.839ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
94722025-11-12T20:49:35.840ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "809f1d68-e7b5-4859-92f0-bcaffad3b4d7", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94732025-11-12T20:49:35.840ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94742025-11-12T20:49:35.842ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 562.155988ms iteration = 9 status = {}
94752025-11-12T20:49:35.842ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "10847cf5-d6bd-46ad-a9ec-2812bbcfc5e0", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94762025-11-12T20:49:35.842ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94772025-11-12T20:49:35.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94782025-11-12T20:49:35.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94792025-11-12T20:49:35.844ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "8ef734ff-2ddf-4b97-a370-5c408d60c444", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94802025-11-12T20:49:35.844ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94812025-11-12T20:49:35.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94822025-11-12T20:49:35.847ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
94832025-11-12T20:49:35.847ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 570.035842ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
94842025-11-12T20:49:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
94852025-11-12T20:49:35.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94862025-11-12T20:49:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94872025-11-12T20:49:35.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94882025-11-12T20:49:35.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94892025-11-12T20:49:35.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94902025-11-12T20:49:35.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94912025-11-12T20:49:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94922025-11-12T20:49:35.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94932025-11-12T20:49:35.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94942025-11-12T20:49:35.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94952025-11-12T20:49:35.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94962025-11-12T20:49:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94972025-11-12T20:49:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94982025-11-12T20:49:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94992025-11-12T20:49:35.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95002025-11-12T20:49:35.882ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
95012025-11-12T20:49:35.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95022025-11-12T20:49:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95032025-11-12T20:49:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95042025-11-12T20:49:35.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95052025-11-12T20:49:35.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95062025-11-12T20:49:35.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95072025-11-12T20:49:35.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95082025-11-12T20:49:35.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95092025-11-12T20:49:35.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95102025-11-12T20:49:35.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
95112025-11-12T20:49:35.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95122025-11-12T20:49:35.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95132025-11-12T20:49:35.899ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
95142025-11-12T20:49:35.899ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95152025-11-12T20:49:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95162025-11-12T20:49:35.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95172025-11-12T20:49:35.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95182025-11-12T20:49:35.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95192025-11-12T20:49:35.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95202025-11-12T20:49:35.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95212025-11-12T20:49:35.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95222025-11-12T20:49:35.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95232025-11-12T20:49:35.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95242025-11-12T20:49:35.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95252025-11-12T20:49:35.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95262025-11-12T20:49:35.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95272025-11-12T20:49:35.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95282025-11-12T20:49:35.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95292025-11-12T20:49:35.916ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
95302025-11-12T20:49:35.916ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
95312025-11-12T20:49:35.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95322025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
95332025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
95342025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95352025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95362025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95372025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
95382025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
95392025-11-12T20:49:35.917ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
95402025-11-12T20:49:35.918ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
95412025-11-12T20:49:35.918ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95422025-11-12T20:49:35.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95432025-11-12T20:49:35.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95442025-11-12T20:49:35.921ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
95452025-11-12T20:49:35.921ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
95462025-11-12T20:49:35.924ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
95472025-11-12T20:49:35.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95482025-11-12T20:49:35.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95492025-11-12T20:49:35.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95502025-11-12T20:49:35.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95512025-11-12T20:49:35.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95522025-11-12T20:49:35.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95532025-11-12T20:49:35.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95542025-11-12T20:49:35.937ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
95552025-11-12T20:49:35.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95562025-11-12T20:49:35.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95572025-11-12T20:49:35.943ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
95582025-11-12T20:49:35.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95592025-11-12T20:49:35.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95602025-11-12T20:49:35.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95612025-11-12T20:49:35.948ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
95622025-11-12T20:49:35.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95632025-11-12T20:49:35.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95642025-11-12T20:49:35.950ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
95652025-11-12T20:49:35.950ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
95662025-11-12T20:49:35.950ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
95672025-11-12T20:49:35.970ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
95682025-11-12T20:49:35.970ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
95692025-11-12T20:49:35.970ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe522ca2-8765-4f64-a8ce-44ea4b68ca36", "content-length": "372", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
95702025-11-12T20:49:35.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95712025-11-12T20:49:35.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95722025-11-12T20:49:35.988ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
95732025-11-12T20:49:35.988ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
95742025-11-12T20:49:35.989ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "71026d9a-5385-4be9-950c-508fe82a7060", "date": "Wed, 12 Nov 2025 20:49:35 GMT"} })
95752025-11-12T20:49:35.989ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95762025-11-12T20:49:35.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95772025-11-12T20:49:35.991ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95782025-11-12T20:49:35.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95792025-11-12T20:49:35.995ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95802025-11-12T20:49:35.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95812025-11-12T20:49:35.999ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
95822025-11-12T20:49:35.999ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
95832025-11-12T20:49:36.000ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4c35df23-7dea-493e-8e89-ac4beab73c79", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
95842025-11-12T20:49:36.003ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 191.70895ms iteration = 14 status = {}
95852025-11-12T20:49:36.022ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
95862025-11-12T20:49:36.022ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
95872025-11-12T20:49:36.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95882025-11-12T20:49:36.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95892025-11-12T20:49:36.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95902025-11-12T20:49:36.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
95912025-11-12T20:49:36.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95922025-11-12T20:49:36.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
95932025-11-12T20:49:36.061ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
95942025-11-12T20:49:36.062ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "30831fc7-752d-47fa-b07a-810fdd7bbfb6", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
95952025-11-12T20:49:36.065ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
95962025-11-12T20:49:36.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = Dns step id = Ensure
95972025-11-12T20:49:36.065ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
95982025-11-12T20:49:36.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
95992025-11-12T20:49:36.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96002025-11-12T20:49:36.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96012025-11-12T20:49:36.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96022025-11-12T20:49:36.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96032025-11-12T20:49:36.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96042025-11-12T20:49:36.072ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
96052025-11-12T20:49:36.072ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
96062025-11-12T20:49:36.072ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
96072025-11-12T20:49:36.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96082025-11-12T20:49:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96092025-11-12T20:49:36.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96102025-11-12T20:49:36.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96112025-11-12T20:49:36.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96122025-11-12T20:49:36.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96132025-11-12T20:49:36.106ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b6f729f9-6419-49bb-bdd0-71731a004b71", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
96142025-11-12T20:49:36.106ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
96152025-11-12T20:49:36.106ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
96162025-11-12T20:49:36.113ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a899c88d-5cba-406e-bd1f-0b8ddb542de5", "content-length": "682", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
96172025-11-12T20:49:36.113ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96182025-11-12T20:49:36.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96192025-11-12T20:49:36.116ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96202025-11-12T20:49:36.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96212025-11-12T20:49:36.119ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
96222025-11-12T20:49:36.119ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
96232025-11-12T20:49:36.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96242025-11-12T20:49:36.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96252025-11-12T20:49:36.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96262025-11-12T20:49:36.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96272025-11-12T20:49:36.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96282025-11-12T20:49:36.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96292025-11-12T20:49:36.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96302025-11-12T20:49:36.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96312025-11-12T20:49:36.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96322025-11-12T20:49:36.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96332025-11-12T20:49:36.130ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
96342025-11-12T20:49:36.130ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
96352025-11-12T20:49:36.130ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
96362025-11-12T20:49:36.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96372025-11-12T20:49:36.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96382025-11-12T20:49:36.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96392025-11-12T20:49:36.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96402025-11-12T20:49:36.132ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 852.516521ms iteration = 9 status = {}
96412025-11-12T20:49:36.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96422025-11-12T20:49:36.137ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96432025-11-12T20:49:36.138ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
96442025-11-12T20:49:36.138ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
96452025-11-12T20:49:36.138ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
96462025-11-12T20:49:36.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96472025-11-12T20:49:36.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96482025-11-12T20:49:36.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96492025-11-12T20:49:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96502025-11-12T20:49:36.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96512025-11-12T20:49:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96522025-11-12T20:49:36.145ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
96532025-11-12T20:49:36.145ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
96542025-11-12T20:49:36.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96552025-11-12T20:49:36.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96562025-11-12T20:49:36.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96572025-11-12T20:49:36.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96582025-11-12T20:49:36.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96592025-11-12T20:49:36.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96602025-11-12T20:49:36.152ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
96612025-11-12T20:49:36.152ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
96622025-11-12T20:49:36.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96632025-11-12T20:49:36.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96642025-11-12T20:49:36.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96652025-11-12T20:49:36.155ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
96662025-11-12T20:49:36.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96672025-11-12T20:49:36.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96682025-11-12T20:49:36.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96692025-11-12T20:49:36.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96702025-11-12T20:49:36.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96712025-11-12T20:49:36.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96722025-11-12T20:49:36.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96732025-11-12T20:49:36.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
96742025-11-12T20:49:36.167ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
96752025-11-12T20:49:36.167ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
96762025-11-12T20:49:36.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = OmicronZones step id = Cleanup
96772025-11-12T20:49:36.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = OmicronZones step id = Cleanup
96782025-11-12T20:49:36.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = PhysicalDisks step id = Cleanup
96792025-11-12T20:49:36.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = Clickhouse step id = Ensure
96802025-11-12T20:49:36.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = Clickhouse step id = Ensure
96812025-11-12T20:49:36.199ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
96822025-11-12T20:49:36.204ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "583a9e78-de57-40fc-9102-e14bec2afe0e", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
96832025-11-12T20:49:36.205ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = SupportBundles step id = Cleanup
96842025-11-12T20:49:36.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96852025-11-12T20:49:36.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96862025-11-12T20:49:36.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96872025-11-12T20:49:36.209ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
96882025-11-12T20:49:36.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96892025-11-12T20:49:36.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96902025-11-12T20:49:36.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96912025-11-12T20:49:36.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96922025-11-12T20:49:36.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96932025-11-12T20:49:36.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
96942025-11-12T20:49:36.220ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = OmicronZones step id = Cleanup
96952025-11-12T20:49:36.220ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
96962025-11-12T20:49:36.220ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
96972025-11-12T20:49:36.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96982025-11-12T20:49:36.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96992025-11-12T20:49:36.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97002025-11-12T20:49:36.223ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
97012025-11-12T20:49:36.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
97022025-11-12T20:49:36.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
97032025-11-12T20:49:36.265ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
97042025-11-12T20:49:36.265ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
97052025-11-12T20:49:36.265ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = Cockroach step id = Ensure
97062025-11-12T20:49:36.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
97072025-11-12T20:49:36.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
97082025-11-12T20:49:36.305ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
97092025-11-12T20:49:36.305ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e761fec2-49db-4d72-84c3-a185bd5260e5 step component = MgsUpdates step id = Ensure
97102025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Dependency
97112025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Dependency
97122025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97132025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97142025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.02752448s iteration = 6 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3080,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":92686938,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":92686888,"secs":0}},"event_index":1,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":92702099,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":205781496,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":205781446,"secs":0}},"event_index":2,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":298523056,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":76323026,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":76322986,"secs":0}},"event_index":3,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":374869162,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":137177500,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":137177450,"secs":0}},"event_index":4,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":512061592,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":275145100,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":275145070,"secs":0}},"event_index":5,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":787219752,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":102719548,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":102719508,"secs":0}},"event_index":6,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889953480,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5940,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5890,"secs":0}},"event_index":7,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889969131,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3080,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3040,"secs":0}},"event_index":8,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889975071,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4050,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4000,"secs":0}},"event_index":9,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889981751,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2500,"secs":0}},"event_index":10,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889988991,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37751767,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":37751057,"secs":0}},"event_index":11,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":927741238,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14435913,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14435823,"secs":0}},"event_index":12,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":942190352,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":45638711,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":45638671,"secs":0}},"event_index":13,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":987843274,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39297907,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":39297877,"secs":0}},"event_index":14,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":27154791,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5651,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5620,"secs":0}},"event_index":15,"execution_id":"2c29da31-607e-40c1-b8da-892bc6e5c4b0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":27171791,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
97152025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97162025-11-12T20:49:36.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
97172025-11-12T20:49:36.307ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97182025-11-12T20:49:36.307ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97192025-11-12T20:49:36.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
97202025-11-12T20:49:36.310ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97212025-11-12T20:49:36.310ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
97222025-11-12T20:49:36.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97232025-11-12T20:49:36.310ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97242025-11-12T20:49:36.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
97252025-11-12T20:49:36.311ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97262025-11-12T20:49:36.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97272025-11-12T20:49:36.313ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97282025-11-12T20:49:36.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
97292025-11-12T20:49:36.315ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97302025-11-12T20:49:36.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97312025-11-12T20:49:36.317ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
97322025-11-12T20:49:36.317ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.618656ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
97332025-11-12T20:49:36.327ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(677c0ea6-1049-4589-9f51-940941493fc6, 0), (4856956c-8555-4ed1-a4d2-383b97af5a6c, 0), (e3b4cbcd-3dac-4af2-a874-38311f43f4d2, 0), (17566f7a-6076-4f56-b7da-c6a2f3173ae2, 0)] collection_id = 677c0ea6-1049-4589-9f51-940941493fc6
97342025-11-12T20:49:36.327ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97352025-11-12T20:49:36.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
97362025-11-12T20:49:36.474ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 677c0ea6-1049-4589-9f51-940941493fc6 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
97372025-11-12T20:49:36.474ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97382025-11-12T20:49:36.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
97392025-11-12T20:49:36.478ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(4856956c-8555-4ed1-a4d2-383b97af5a6c, 0), (e3b4cbcd-3dac-4af2-a874-38311f43f4d2, 0), (17566f7a-6076-4f56-b7da-c6a2f3173ae2, 0), (49827f1a-3808-4b86-9622-c369fde9d0c0, 0)] collection_id = 4856956c-8555-4ed1-a4d2-383b97af5a6c
97402025-11-12T20:49:36.478ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97412025-11-12T20:49:36.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
97422025-11-12T20:49:36.585ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 4856956c-8555-4ed1-a4d2-383b97af5a6c file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
97432025-11-12T20:49:36.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97442025-11-12T20:49:36.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
97452025-11-12T20:49:36.589ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(e3b4cbcd-3dac-4af2-a874-38311f43f4d2, 0), (17566f7a-6076-4f56-b7da-c6a2f3173ae2, 0), (49827f1a-3808-4b86-9622-c369fde9d0c0, 0)]
97462025-11-12T20:49:36.590ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1172, tv_nsec: 84761353 } })
97472025-11-12T20:49:36.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1172, tv_nsec: 118880327 } })
97482025-11-12T20:49:36.625ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
97492025-11-12T20:49:36.625ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
97502025-11-12T20:49:36.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
97512025-11-12T20:49:36.625ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f166c1d-0b76-4552-b0ee-eabdb9103295", "content-length": "476", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97522025-11-12T20:49:36.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
97532025-11-12T20:49:36.626ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24daa6f4-d214-4379-83e7-9af1d1ebb6ca", "content-length": "734", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97542025-11-12T20:49:36.626ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
97552025-11-12T20:49:36.626ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ed7887a-43f1-4135-8ca1-32bf6b8c7038", "content-length": "111", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97562025-11-12T20:49:36.626ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
97572025-11-12T20:49:36.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "05fb203e-85b9-4a76-b7d0-6e5562334fa3", "content-length": "111", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97582025-11-12T20:49:36.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
97592025-11-12T20:49:36.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "20a65486-edd4-4d7c-940a-b6e53d78ba3a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97602025-11-12T20:49:36.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
97612025-11-12T20:49:36.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0c33c11-9e05-4f73-a6bb-0a7f6496b868", "content-length": "163", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97622025-11-12T20:49:36.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
97632025-11-12T20:49:36.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2dbcbbd-32da-46da-8bcc-3eaee2f57014", "content-length": "166", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97642025-11-12T20:49:36.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
97652025-11-12T20:49:36.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7425cf72-1f51-4789-9e2a-649e950fd89a", "content-length": "166", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97662025-11-12T20:49:36.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
97672025-11-12T20:49:36.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebad944b-cdc0-498d-add6-07b59f3024c1", "content-length": "702", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97682025-11-12T20:49:36.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
97692025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f033851-bd5c-422f-b213-8aa37207a4f7", "content-length": "727", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97702025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
97712025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6ac90e4-fdeb-4937-b0e2-570b6c7ca085", "content-length": "729", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97722025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
97732025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "664f540c-d9db-44ff-8de3-afc4bbbd626a", "content-length": "728", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97742025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
97752025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "45aec4bb-835f-423c-9e78-a5ccc34311ab", "content-length": "722", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97762025-11-12T20:49:36.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
97772025-11-12T20:49:36.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "2871406e-b4dc-4abe-88dd-6ca56eab8d82", "content-length": "10", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97782025-11-12T20:49:36.631ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
97792025-11-12T20:49:36.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cef76c55-125d-4c14-91ef-34307bfbe681", "content-length": "144", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97802025-11-12T20:49:36.631ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
97812025-11-12T20:49:36.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37e31ae5-ad2d-4a85-b422-b3da11a7d7c2", "content-length": "147", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97822025-11-12T20:49:36.631ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
97832025-11-12T20:49:36.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60fdd019-0ac3-4d74-a061-8ee9b41ff0af", "content-length": "109", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97842025-11-12T20:49:36.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
97852025-11-12T20:49:36.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e984d20-fc3b-454d-b185-559805b3832d", "content-length": "109", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97862025-11-12T20:49:36.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
97872025-11-12T20:49:36.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc151b56-f530-45d3-b6d9-d6b6320cbe59", "content-length": "163", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97882025-11-12T20:49:36.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
97892025-11-12T20:49:36.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4bddd4e-5f57-454c-9258-21f9a115404a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97902025-11-12T20:49:36.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
97912025-11-12T20:49:36.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "525ab994-0b50-4074-80df-6f7be0c36c04", "content-length": "166", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97922025-11-12T20:49:36.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
97932025-11-12T20:49:36.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b1f4a60-5447-488b-a59d-bead82e32d1d", "content-length": "166", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97942025-11-12T20:49:36.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
97952025-11-12T20:49:36.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "35e72df5-62e5-4bba-928a-718922c2f333", "content-length": "702", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97962025-11-12T20:49:36.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
97972025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f023389-3460-4157-a361-c943141afb6d", "content-length": "727", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
97982025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
97992025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a699af97-d4b8-48c8-bec8-52d3f69ff89b", "content-length": "729", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98002025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
98012025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29fa364c-8bc8-4328-8476-047d3c79db7c", "content-length": "728", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98022025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98032025-11-12T20:49:36.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
98042025-11-12T20:49:36.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98052025-11-12T20:49:36.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
98062025-11-12T20:49:36.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98072025-11-12T20:49:36.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
98082025-11-12T20:49:36.641ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98092025-11-12T20:49:36.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
98102025-11-12T20:49:36.643ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98112025-11-12T20:49:36.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
98122025-11-12T20:49:36.676ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
98132025-11-12T20:49:36.676ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
98142025-11-12T20:49:36.780ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d8cc77c-b15e-4789-a127-2575b6f7d20f", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98152025-11-12T20:49:36.783ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
98162025-11-12T20:49:36.783ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
98172025-11-12T20:49:36.783ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
98182025-11-12T20:49:36.783ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
98192025-11-12T20:49:36.783ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
98202025-11-12T20:49:36.783ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
98212025-11-12T20:49:36.783ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
98222025-11-12T20:49:36.783ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
98232025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "69956171-67dd-4646-8dd2-56f796319a2f", "content-length": "64", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98242025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
98252025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "df9bf81c-0d0a-4215-975c-5af12ce2e6d4", "content-length": "64", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98262025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6e0e466-44e4-47e3-8480-392a230fca65", "content-length": "64", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98272025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9630883b-2233-46f5-91e0-1fa2bb9292a2", "content-length": "64", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98282025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
98292025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
98302025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
98312025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ede1992a-ff3a-4139-b6ff-1459ececaa06", "content-length": "64", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98322025-11-12T20:49:36.784ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
98332025-11-12T20:49:36.822ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f66ada13-936b-44e4-a5db-fc9077563d5c", "content-length": "421215", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98342025-11-12T20:49:36.823ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "810f0c63-f697-4944-817e-73ebb65c5cc3", "content-length": "392984", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98352025-11-12T20:49:36.824ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "13ba1916-2637-4674-b49e-9d7a0afd5f85", "content-length": "389591", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98362025-11-12T20:49:36.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed721cf3-2ad4-4847-921b-992d021ee6fe", "content-length": "387956", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98372025-11-12T20:49:36.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bce15fac-23e9-4449-b7d1-ebaca3cc33da", "content-length": "389358", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98382025-11-12T20:49:36.829ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
98392025-11-12T20:49:36.829ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
98402025-11-12T20:49:36.829ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
98412025-11-12T20:49:36.829ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
98422025-11-12T20:49:36.829ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
98432025-11-12T20:49:36.859ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
98442025-11-12T20:49:36.859ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
98452025-11-12T20:49:36.863ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5c1bccb-9b60-4db3-8d5d-d1c292aabe3f", "content-length": "121", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98462025-11-12T20:49:36.864ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
98472025-11-12T20:49:36.956ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
98482025-11-12T20:49:36.957ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
98492025-11-12T20:49:36.957ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d10305c4-2c7c-4f40-a986-8ffaca947a18", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98502025-11-12T20:49:36.957ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
98512025-11-12T20:49:36.959ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
98522025-11-12T20:49:36.960ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
98532025-11-12T20:49:36.960ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "125ffcdf-fe45-406e-8276-885a5539e052", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98542025-11-12T20:49:36.960ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
98552025-11-12T20:49:36.963ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
98562025-11-12T20:49:36.963ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
98572025-11-12T20:49:36.964ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "28e38258-d3a4-4fb3-98ea-ba63831f55d5", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:49:36 GMT"} })
98582025-11-12T20:49:36.964ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
98592025-11-12T20:49:36.966ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
98602025-11-12T20:49:36.966ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
98612025-11-12T20:49:36.968ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98622025-11-12T20:49:36.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
98632025-11-12T20:49:36.970ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98642025-11-12T20:49:36.973ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
98652025-11-12T20:49:36.973ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98662025-11-12T20:49:36.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
98672025-11-12T20:49:37.176ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
98682025-11-12T20:49:37.176ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 time_started = 2025-11-12 20:49:36.625152 UTC
98692025-11-12T20:49:37.176ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 23 reason = Dependency
98702025-11-12T20:49:37.176ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98712025-11-12T20:49:37.176ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 871.742029ms iteration = 6 status = {"collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","time_done":"2025-11-12 20:49:36.966630 UTC","time_started":"2025-11-12 20:49:36.625152 UTC"}
98722025-11-12T20:49:37.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98732025-11-12T20:49:37.178ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98742025-11-12T20:49:37.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
98752025-11-12T20:49:37.182ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98762025-11-12T20:49:37.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98772025-11-12T20:49:37.185ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98782025-11-12T20:49:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98792025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 62.506277ms iteration = 23 status = {"Loaded":{"collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","time_loaded":"2025-11-12T20:49:37.176860843Z","time_started":"2025-11-12T20:49:36.625152Z"}}
98802025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 8 reason = Dependency
98812025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Dependency
98822025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Dependency
98832025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98842025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
98852025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
98862025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98872025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
98882025-11-12T20:49:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
98892025-11-12T20:49:37.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98902025-11-12T20:49:37.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98912025-11-12T20:49:37.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98922025-11-12T20:49:37.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
98932025-11-12T20:49:37.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98942025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
98952025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
98962025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98972025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
98982025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
98992025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99002025-11-12T20:49:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99012025-11-12T20:49:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99022025-11-12T20:49:37.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99032025-11-12T20:49:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99042025-11-12T20:49:37.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99052025-11-12T20:49:37.248ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99062025-11-12T20:49:37.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
99072025-11-12T20:49:37.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99082025-11-12T20:49:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99092025-11-12T20:49:37.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
99102025-11-12T20:49:37.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99112025-11-12T20:49:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99122025-11-12T20:49:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
99132025-11-12T20:49:37.251ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99142025-11-12T20:49:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99152025-11-12T20:49:37.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99162025-11-12T20:49:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99172025-11-12T20:49:37.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99182025-11-12T20:49:37.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99192025-11-12T20:49:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99202025-11-12T20:49:37.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99212025-11-12T20:49:37.256ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
99222025-11-12T20:49:37.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99232025-11-12T20:49:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99242025-11-12T20:49:37.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99252025-11-12T20:49:37.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
99262025-11-12T20:49:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99272025-11-12T20:49:37.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99282025-11-12T20:49:37.261ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
99292025-11-12T20:49:37.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99302025-11-12T20:49:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99312025-11-12T20:49:37.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99322025-11-12T20:49:37.263ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
99332025-11-12T20:49:37.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99342025-11-12T20:49:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99352025-11-12T20:49:37.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
99362025-11-12T20:49:37.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99372025-11-12T20:49:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99382025-11-12T20:49:37.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99392025-11-12T20:49:37.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
99402025-11-12T20:49:37.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99412025-11-12T20:49:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99422025-11-12T20:49:37.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99432025-11-12T20:49:37.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
99442025-11-12T20:49:37.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
99452025-11-12T20:49:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
99462025-11-12T20:49:37.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99472025-11-12T20:49:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99482025-11-12T20:49:37.276ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
99492025-11-12T20:49:37.276ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 36.74046ms iteration = 8 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
99502025-11-12T20:49:37.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99512025-11-12T20:49:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99522025-11-12T20:49:37.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99532025-11-12T20:49:37.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
99542025-11-12T20:49:37.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99552025-11-12T20:49:37.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99562025-11-12T20:49:37.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99572025-11-12T20:49:37.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99582025-11-12T20:49:37.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99592025-11-12T20:49:37.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
99602025-11-12T20:49:37.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99612025-11-12T20:49:37.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99622025-11-12T20:49:37.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99632025-11-12T20:49:37.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
99642025-11-12T20:49:37.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99652025-11-12T20:49:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99662025-11-12T20:49:37.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99672025-11-12T20:49:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99682025-11-12T20:49:37.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99692025-11-12T20:49:37.303ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 64.122419ms iteration = 16 status = {"physical_disks_added":0}
99702025-11-12T20:49:37.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
99712025-11-12T20:49:37.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99722025-11-12T20:49:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99732025-11-12T20:49:37.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99742025-11-12T20:49:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99752025-11-12T20:49:37.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99762025-11-12T20:49:37.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
99772025-11-12T20:49:37.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99782025-11-12T20:49:37.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99792025-11-12T20:49:37.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99802025-11-12T20:49:37.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99812025-11-12T20:49:37.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99822025-11-12T20:49:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
99832025-11-12T20:49:37.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99842025-11-12T20:49:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99852025-11-12T20:49:37.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99862025-11-12T20:49:37.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99872025-11-12T20:49:37.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99882025-11-12T20:49:37.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99892025-11-12T20:49:37.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99902025-11-12T20:49:37.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
99912025-11-12T20:49:37.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99922025-11-12T20:49:37.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99932025-11-12T20:49:37.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99942025-11-12T20:49:37.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
99952025-11-12T20:49:37.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99962025-11-12T20:49:37.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99972025-11-12T20:49:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99982025-11-12T20:49:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99992025-11-12T20:49:37.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100002025-11-12T20:49:37.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100012025-11-12T20:49:37.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100022025-11-12T20:49:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
100032025-11-12T20:49:37.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100042025-11-12T20:49:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100052025-11-12T20:49:37.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100062025-11-12T20:49:37.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
100072025-11-12T20:49:37.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100082025-11-12T20:49:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100092025-11-12T20:49:37.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100102025-11-12T20:49:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100112025-11-12T20:49:37.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100122025-11-12T20:49:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100132025-11-12T20:49:37.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100142025-11-12T20:49:37.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
100152025-11-12T20:49:37.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100162025-11-12T20:49:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100172025-11-12T20:49:37.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100182025-11-12T20:49:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100192025-11-12T20:49:37.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100202025-11-12T20:49:37.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
100212025-11-12T20:49:37.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100222025-11-12T20:49:37.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100232025-11-12T20:49:37.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100242025-11-12T20:49:37.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100252025-11-12T20:49:37.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100262025-11-12T20:49:37.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
100272025-11-12T20:49:37.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100282025-11-12T20:49:37.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100292025-11-12T20:49:37.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100302025-11-12T20:49:37.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100312025-11-12T20:49:37.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100322025-11-12T20:49:37.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
100332025-11-12T20:49:37.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100342025-11-12T20:49:37.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100352025-11-12T20:49:37.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100362025-11-12T20:49:37.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100372025-11-12T20:49:37.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100382025-11-12T20:49:37.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
100392025-11-12T20:49:37.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100402025-11-12T20:49:37.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100412025-11-12T20:49:37.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100422025-11-12T20:49:37.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100432025-11-12T20:49:37.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100442025-11-12T20:49:37.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
100452025-11-12T20:49:37.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100462025-11-12T20:49:37.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100472025-11-12T20:49:37.461ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
100482025-11-12T20:49:37.461ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
100492025-11-12T20:49:37.461ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = d9110492-9e38-4c14-9905-ae8362d4e096 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
100502025-11-12T20:49:37.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100512025-11-12T20:49:37.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100522025-11-12T20:49:37.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100532025-11-12T20:49:37.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
100542025-11-12T20:49:37.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100552025-11-12T20:49:37.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100562025-11-12T20:49:37.470ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
100572025-11-12T20:49:37.470ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
100582025-11-12T20:49:37.470ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 230.826393ms iteration = 14 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
100592025-11-12T20:49:37.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 297 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = fa0b91c0-1704-4dad-8c45-7cf3ab100c84 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
100602025-11-12T20:49:40.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 50 reason = Timeout
100612025-11-12T20:49:40.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
100622025-11-12T20:49:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
100632025-11-12T20:49:40.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100642025-11-12T20:49:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
100652025-11-12T20:49:40.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
100662025-11-12T20:49:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
100672025-11-12T20:49:40.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
100682025-11-12T20:49:40.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.663963ms iteration = 50 status = {"config_updated":false}
100692025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
100702025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100712025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
100722025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 51 reason = Timeout
100732025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100742025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
100752025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100762025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
100772025-11-12T20:49:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
100782025-11-12T20:49:45.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100792025-11-12T20:49:45.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100802025-11-12T20:49:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
100812025-11-12T20:49:45.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
100822025-11-12T20:49:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
100832025-11-12T20:49:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
100842025-11-12T20:49:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100852025-11-12T20:49:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
100862025-11-12T20:49:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100872025-11-12T20:49:45.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
100882025-11-12T20:49:45.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.342576ms iteration = 51 status = {"config_updated":false}
100892025-11-12T20:49:45.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
100902025-11-12T20:49:45.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100912025-11-12T20:49:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
100922025-11-12T20:49:45.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100932025-11-12T20:49:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
100942025-11-12T20:49:45.305ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
100952025-11-12T20:49:45.305ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.643693ms iteration = 26 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
100962025-11-12T20:49:45.314ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100972025-11-12T20:49:45.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100982025-11-12T20:49:45.316ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.383472ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
100992025-11-12T20:49:47.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 329 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 69a44d39-dc30-4a08-bd58-a8638db27ef4 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
101002025-11-12T20:49:50.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 24 reason = Timeout
101012025-11-12T20:49:50.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
101022025-11-12T20:49:50.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
101032025-11-12T20:49:50.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 52 reason = Timeout
101042025-11-12T20:49:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101052025-11-12T20:49:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101062025-11-12T20:49:50.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101072025-11-12T20:49:50.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101082025-11-12T20:49:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
101092025-11-12T20:49:50.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
101102025-11-12T20:49:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
101112025-11-12T20:49:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
101122025-11-12T20:49:50.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101132025-11-12T20:49:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101142025-11-12T20:49:50.286ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 original_time_started = 2025-11-12 20:49:36.625152 UTC
101152025-11-12T20:49:50.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.142314ms iteration = 24 status = {"Loaded":{"collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","time_loaded":"2025-11-12T20:49:50.279281923Z","time_started":"2025-11-12T20:49:36.625152Z"}}
101162025-11-12T20:49:50.287ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 18 reason = Timeout
101172025-11-12T20:49:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
101182025-11-12T20:49:50.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
101192025-11-12T20:49:50.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.579396ms iteration = 52 status = {"config_updated":false}
101202025-11-12T20:49:50.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
101212025-11-12T20:49:50.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101222025-11-12T20:49:50.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
101232025-11-12T20:49:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
101242025-11-12T20:49:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
101252025-11-12T20:49:50.292ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.363459ms iteration = 18 status = "NoSitrep"
101262025-11-12T20:49:50.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 18 reason = Timeout
101272025-11-12T20:49:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
101282025-11-12T20:49:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
101292025-11-12T20:49:50.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
101302025-11-12T20:49:50.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
101312025-11-12T20:49:50.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101322025-11-12T20:49:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
101332025-11-12T20:49:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
101342025-11-12T20:49:50.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
101352025-11-12T20:49:50.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
101362025-11-12T20:49:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
101372025-11-12T20:49:50.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101382025-11-12T20:49:50.303ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
101392025-11-12T20:49:50.303ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
101402025-11-12T20:49:50.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.214454ms iteration = 18 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
101412025-11-12T20:49:55.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
101422025-11-12T20:49:55.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
101432025-11-12T20:49:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101442025-11-12T20:49:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101452025-11-12T20:49:55.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 53 reason = Timeout
101462025-11-12T20:49:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101472025-11-12T20:49:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101482025-11-12T20:49:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101492025-11-12T20:49:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101502025-11-12T20:49:55.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101512025-11-12T20:49:55.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101522025-11-12T20:49:55.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101532025-11-12T20:49:55.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101542025-11-12T20:49:55.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.639544ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
101552025-11-12T20:49:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
101562025-11-12T20:49:55.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101572025-11-12T20:49:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101582025-11-12T20:49:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
101592025-11-12T20:49:55.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101602025-11-12T20:49:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101612025-11-12T20:49:55.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
101622025-11-12T20:49:55.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.442023ms iteration = 53 status = {"config_updated":false}
101632025-11-12T20:49:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101642025-11-12T20:49:55.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101652025-11-12T20:49:55.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101662025-11-12T20:49:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
101672025-11-12T20:49:55.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101682025-11-12T20:49:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101692025-11-12T20:49:55.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
101702025-11-12T20:49:55.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.18767ms iteration = 27 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
101712025-11-12T20:49:57.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 293 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = ef58d4f6-810c-4baf-b3dd-ee1698d220b9 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
101722025-11-12T20:50:00.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 54 reason = Timeout
101732025-11-12T20:50:00.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101742025-11-12T20:50:00.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101752025-11-12T20:50:00.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101762025-11-12T20:50:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
101772025-11-12T20:50:00.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101782025-11-12T20:50:00.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101792025-11-12T20:50:00.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
101802025-11-12T20:50:00.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.689383ms iteration = 54 status = {"config_updated":false}
101812025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
101822025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 10 reason = Timeout
101832025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
101842025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
101852025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
101862025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
101872025-11-12T20:50:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
101882025-11-12T20:50:05.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 25 reason = Timeout
101892025-11-12T20:50:05.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
101902025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
101912025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
101922025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
101932025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
101942025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101952025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
101962025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
101972025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101982025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
101992025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
102002025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102012025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
102022025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
102032025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
102042025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102052025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
102062025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
102072025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
102082025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
102092025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
102102025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
102112025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
102122025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
102132025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
102142025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
102152025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 10 reason = Timeout
102162025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
102172025-11-12T20:50:05.280ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
102182025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102192025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
102202025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
102212025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
102222025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102232025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102242025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
102252025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 55 reason = Timeout
102262025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102272025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
102282025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102292025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
102302025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
102312025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102322025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102332025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
102342025-11-12T20:50:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102352025-11-12T20:50:05.281ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1200, tv_nsec: 775549941 } })
102362025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
102372025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
102382025-11-12T20:50:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1200, tv_nsec: 775591942 } })
102392025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
102402025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
102412025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102422025-11-12T20:50:05.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1200, tv_nsec: 775624793 } })
102432025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
102442025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
102452025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
102462025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102472025-11-12T20:50:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
102482025-11-12T20:50:05.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
102492025-11-12T20:50:05.282ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102502025-11-12T20:50:05.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102512025-11-12T20:50:05.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102522025-11-12T20:50:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102532025-11-12T20:50:05.283ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.979564ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
102542025-11-12T20:50:05.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.324986ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
102552025-11-12T20:50:05.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102562025-11-12T20:50:05.284ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102572025-11-12T20:50:05.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102582025-11-12T20:50:05.285ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102592025-11-12T20:50:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102602025-11-12T20:50:05.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 19 reason = Timeout
102612025-11-12T20:50:05.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
102622025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
102632025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102642025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102652025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102662025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
102672025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102682025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102692025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102702025-11-12T20:50:05.287ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102712025-11-12T20:50:05.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102722025-11-12T20:50:05.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102732025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102742025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102752025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
102762025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
102772025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
102782025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
102792025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
102802025-11-12T20:50:05.291ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102812025-11-12T20:50:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
102822025-11-12T20:50:05.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 19 reason = Timeout
102832025-11-12T20:50:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102842025-11-12T20:50:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
102852025-11-12T20:50:05.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102862025-11-12T20:50:05.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102872025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
102882025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
102892025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
102902025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102912025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102922025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
102932025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
102942025-11-12T20:50:05.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102952025-11-12T20:50:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
102962025-11-12T20:50:05.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102972025-11-12T20:50:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102982025-11-12T20:50:05.294ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
102992025-11-12T20:50:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.223668ms iteration = 55 status = {"config_updated":false}
103002025-11-12T20:50:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
103012025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
103022025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
103032025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 14.755502ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
103042025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
103052025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
103062025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103072025-11-12T20:50:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103082025-11-12T20:50:05.296ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103092025-11-12T20:50:05.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103102025-11-12T20:50:05.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
103112025-11-12T20:50:05.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
103122025-11-12T20:50:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103132025-11-12T20:50:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
103142025-11-12T20:50:05.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103152025-11-12T20:50:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 13.225442ms iteration = 19 status = "NoSitrep"
103162025-11-12T20:50:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
103172025-11-12T20:50:05.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
103182025-11-12T20:50:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103192025-11-12T20:50:05.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103202025-11-12T20:50:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
103212025-11-12T20:50:05.304ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 original_time_started = 2025-11-12 20:49:36.625152 UTC
103222025-11-12T20:50:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 24.540955ms iteration = 25 status = {"Loaded":{"collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","time_loaded":"2025-11-12T20:50:05.279542866Z","time_started":"2025-11-12T20:49:36.625152Z"}}
103232025-11-12T20:50:05.304ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1200, tv_nsec: 799269755 } })
103242025-11-12T20:50:05.305ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103252025-11-12T20:50:05.305ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
103262025-11-12T20:50:05.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103272025-11-12T20:50:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
103282025-11-12T20:50:05.306ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
103292025-11-12T20:50:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
103302025-11-12T20:50:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
103312025-11-12T20:50:05.307ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
103322025-11-12T20:50:05.307ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 27.128752ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
103332025-11-12T20:50:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
103342025-11-12T20:50:05.308ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103352025-11-12T20:50:05.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103362025-11-12T20:50:05.311ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
103372025-11-12T20:50:05.311ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
103382025-11-12T20:50:05.311ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 31.203598ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
103392025-11-12T20:50:05.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
103402025-11-12T20:50:05.312ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
103412025-11-12T20:50:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
103422025-11-12T20:50:05.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103432025-11-12T20:50:05.313ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.741397ms iteration = 10 status = 0
103442025-11-12T20:50:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
103452025-11-12T20:50:05.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
103462025-11-12T20:50:05.315ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103472025-11-12T20:50:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
103482025-11-12T20:50:05.318ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
103492025-11-12T20:50:05.318ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 25.975572ms iteration = 19 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
103502025-11-12T20:50:05.336ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
103512025-11-12T20:50:05.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.132492ms iteration = 28 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
103522025-11-12T20:50:05.354ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 74.58644ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
103532025-11-12T20:50:05.364ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 84.323392ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
103542025-11-12T20:50:05.365ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
103552025-11-12T20:50:05.365ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
103562025-11-12T20:50:05.365ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
103572025-11-12T20:50:05.365ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103582025-11-12T20:50:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
103592025-11-12T20:50:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
103602025-11-12T20:50:05.366ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
103612025-11-12T20:50:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
103622025-11-12T20:50:05.367ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
103632025-11-12T20:50:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
103642025-11-12T20:50:05.368ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.906305ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
103652025-11-12T20:50:05.368ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
103662025-11-12T20:50:05.368ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
103672025-11-12T20:50:05.368ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
103682025-11-12T20:50:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
103692025-11-12T20:50:05.372ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103702025-11-12T20:50:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103712025-11-12T20:50:05.384ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103722025-11-12T20:50:05.384ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 104.221588ms iteration = 17 status = {"physical_disks_added":0}
103732025-11-12T20:50:05.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103742025-11-12T20:50:05.386ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103752025-11-12T20:50:05.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103762025-11-12T20:50:05.390ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103772025-11-12T20:50:05.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103782025-11-12T20:50:05.392ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103792025-11-12T20:50:05.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103802025-11-12T20:50:05.395ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103812025-11-12T20:50:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103822025-11-12T20:50:05.417ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 136.662547ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
103832025-11-12T20:50:05.417ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.509888ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
103842025-11-12T20:50:05.421ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
103852025-11-12T20:50:05.421ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
103862025-11-12T20:50:05.421ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 141.082701ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
103872025-11-12T20:50:05.528ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
103882025-11-12T20:50:05.529ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 252.073103ms iteration = 10 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
103892025-11-12T20:50:05.571ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
103902025-11-12T20:50:05.572ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
103912025-11-12T20:50:05.572ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "94530228-b006-46c2-9c35-c8ef68ca4dde", "content-length": "26", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
103922025-11-12T20:50:05.572ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
103932025-11-12T20:50:05.574ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
103942025-11-12T20:50:05.575ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
103952025-11-12T20:50:05.576ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e832f36d-c96f-4d48-bc52-4176f274c663", "content-length": "691", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
103962025-11-12T20:50:05.576ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
103972025-11-12T20:50:05.576ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
103982025-11-12T20:50:05.576ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
103992025-11-12T20:50:05.578ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 297.751415ms iteration = 10 status = {}
104002025-11-12T20:50:05.581ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
104012025-11-12T20:50:05.581ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "32c625bc-bf9f-4b95-8d92-7a6c15c5c40b", "content-length": "78", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104022025-11-12T20:50:05.581ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
104032025-11-12T20:50:05.581ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
104042025-11-12T20:50:05.581ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
104052025-11-12T20:50:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
104062025-11-12T20:50:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
104072025-11-12T20:50:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
104082025-11-12T20:50:05.582ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
104092025-11-12T20:50:05.582ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
104102025-11-12T20:50:05.583ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9127fc92-57df-4c4a-82ea-7dbaa2005be7", "content-length": "26", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104112025-11-12T20:50:05.583ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
104122025-11-12T20:50:05.583ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
104132025-11-12T20:50:05.583ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
104142025-11-12T20:50:05.584ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
104152025-11-12T20:50:05.584ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
104162025-11-12T20:50:05.584ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
104172025-11-12T20:50:05.584ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
104182025-11-12T20:50:05.585ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "005b4770-fa82-435d-897b-d155479621f1", "content-length": "372", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104192025-11-12T20:50:05.585ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104202025-11-12T20:50:05.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104212025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
104222025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
104232025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
104242025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
104252025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
104262025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
104272025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
104282025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
104292025-11-12T20:50:05.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
104302025-11-12T20:50:05.588ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
104312025-11-12T20:50:05.588ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
104322025-11-12T20:50:05.588ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
104332025-11-12T20:50:05.588ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
104342025-11-12T20:50:05.589ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "04b26351-382c-4f7e-a39e-9d08264aac51", "content-length": "81", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104352025-11-12T20:50:05.589ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
104362025-11-12T20:50:05.589ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
104372025-11-12T20:50:05.589ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "617418ba-5e25-447d-8c63-e5c28b5bd40a", "content-length": "81", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104382025-11-12T20:50:05.589ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "48fbdbd5-dcba-439d-a854-418666b4667b", "content-length": "81", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104392025-11-12T20:50:05.591ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 310.70757ms iteration = 10 status = {"disabled":false,"errors":[],"sps":[]}
104402025-11-12T20:50:05.598ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
104412025-11-12T20:50:05.598ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
104422025-11-12T20:50:05.598ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c01a8641-2c80-4bc9-ab07-0cc3104b4d39", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104432025-11-12T20:50:05.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104442025-11-12T20:50:05.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104452025-11-12T20:50:05.600ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104462025-11-12T20:50:05.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104472025-11-12T20:50:05.603ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104482025-11-12T20:50:05.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104492025-11-12T20:50:05.606ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
104502025-11-12T20:50:05.606ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
104512025-11-12T20:50:05.606ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5a3cce96-5d66-4dab-85c2-2f6870ff5baa", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104522025-11-12T20:50:05.608ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 328.247935ms iteration = 15 status = {}
104532025-11-12T20:50:05.639ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 133909145 } })
104542025-11-12T20:50:05.644ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 367.526042ms iteration = 10 status = {}
104552025-11-12T20:50:05.706ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104562025-11-12T20:50:05.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104572025-11-12T20:50:05.708ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
104582025-11-12T20:50:05.708ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104592025-11-12T20:50:05.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104602025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
104612025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
104622025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
104632025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
104642025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104652025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104662025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104672025-11-12T20:50:05.718ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
104682025-11-12T20:50:05.719ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
104692025-11-12T20:50:05.719ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
104702025-11-12T20:50:05.719ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
104712025-11-12T20:50:05.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104722025-11-12T20:50:05.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104732025-11-12T20:50:05.722ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
104742025-11-12T20:50:05.722ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
104752025-11-12T20:50:05.895ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e668c362-ce72-4573-9e87-479ab229b080", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104762025-11-12T20:50:05.895ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
104772025-11-12T20:50:05.895ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
104782025-11-12T20:50:05.895ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "21a10806-ae59-4d43-8860-8098a5d90feb", "content-length": "682", "date": "Wed, 12 Nov 2025 20:50:05 GMT"} })
104792025-11-12T20:50:05.895ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104802025-11-12T20:50:05.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104812025-11-12T20:50:05.899ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104822025-11-12T20:50:05.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104832025-11-12T20:50:05.901ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104842025-11-12T20:50:05.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104852025-11-12T20:50:05.903ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104862025-11-12T20:50:05.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104872025-11-12T20:50:05.910ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 629.5003ms iteration = 10 status = {}
104882025-11-12T20:50:07.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 281 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = ce71df84-28cf-4226-9df5-000447e06f62 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
104892025-11-12T20:50:10.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 56 reason = Timeout
104902025-11-12T20:50:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
104912025-11-12T20:50:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
104922025-11-12T20:50:10.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104932025-11-12T20:50:10.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
104942025-11-12T20:50:10.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
104952025-11-12T20:50:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
104962025-11-12T20:50:10.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
104972025-11-12T20:50:10.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.159336ms iteration = 56 status = {"config_updated":false}
104982025-11-12T20:50:15.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
104992025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
105002025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 57 reason = Timeout
105012025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105022025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105032025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105042025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105052025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105062025-11-12T20:50:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105072025-11-12T20:50:15.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105082025-11-12T20:50:15.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105092025-11-12T20:50:15.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105102025-11-12T20:50:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105112025-11-12T20:50:15.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.26411ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
105122025-11-12T20:50:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
105132025-11-12T20:50:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
105142025-11-12T20:50:15.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105152025-11-12T20:50:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105162025-11-12T20:50:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105172025-11-12T20:50:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105182025-11-12T20:50:15.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
105192025-11-12T20:50:15.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.571986ms iteration = 57 status = {"config_updated":false}
105202025-11-12T20:50:15.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105212025-11-12T20:50:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105222025-11-12T20:50:15.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105232025-11-12T20:50:15.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
105242025-11-12T20:50:15.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105252025-11-12T20:50:15.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105262025-11-12T20:50:15.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
105272025-11-12T20:50:15.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.986507ms iteration = 29 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
105282025-11-12T20:50:17.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 418 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 65642b18-4ed7-483c-9481-6daa8a8306fd response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
105292025-11-12T20:50:20.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 26 reason = Timeout
105302025-11-12T20:50:20.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
105312025-11-12T20:50:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
105322025-11-12T20:50:20.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 58 reason = Timeout
105332025-11-12T20:50:20.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105342025-11-12T20:50:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105352025-11-12T20:50:20.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105362025-11-12T20:50:20.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105372025-11-12T20:50:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
105382025-11-12T20:50:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
105392025-11-12T20:50:20.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105402025-11-12T20:50:20.284ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
105412025-11-12T20:50:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105422025-11-12T20:50:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
105432025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
105442025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.526949ms iteration = 58 status = {"config_updated":false}
105452025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 original_time_started = 2025-11-12 20:49:36.625152 UTC
105462025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.37302ms iteration = 26 status = {"Loaded":{"collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","time_loaded":"2025-11-12T20:50:20.279058450Z","time_started":"2025-11-12T20:49:36.625152Z"}}
105472025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 20 reason = Timeout
105482025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
105492025-11-12T20:50:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
105502025-11-12T20:50:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105512025-11-12T20:50:20.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
105522025-11-12T20:50:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
105532025-11-12T20:50:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
105542025-11-12T20:50:20.292ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.363499ms iteration = 20 status = "NoSitrep"
105552025-11-12T20:50:20.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 20 reason = Timeout
105562025-11-12T20:50:20.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
105572025-11-12T20:50:20.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
105582025-11-12T20:50:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
105592025-11-12T20:50:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
105602025-11-12T20:50:20.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105612025-11-12T20:50:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
105622025-11-12T20:50:20.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
105632025-11-12T20:50:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
105642025-11-12T20:50:20.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
105652025-11-12T20:50:20.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
105662025-11-12T20:50:20.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105672025-11-12T20:50:20.303ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
105682025-11-12T20:50:20.304ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
105692025-11-12T20:50:20.304ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.658406ms iteration = 20 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
105702025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
105712025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
105722025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105732025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105742025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 59 reason = Timeout
105752025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105762025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105772025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105782025-11-12T20:50:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105792025-11-12T20:50:25.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105802025-11-12T20:50:25.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105812025-11-12T20:50:25.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105822025-11-12T20:50:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105832025-11-12T20:50:25.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.297841ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
105842025-11-12T20:50:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
105852025-11-12T20:50:25.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105862025-11-12T20:50:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
105872025-11-12T20:50:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105882025-11-12T20:50:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105892025-11-12T20:50:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105902025-11-12T20:50:25.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
105912025-11-12T20:50:25.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.604553ms iteration = 59 status = {"config_updated":false}
105922025-11-12T20:50:25.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105932025-11-12T20:50:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105942025-11-12T20:50:25.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105952025-11-12T20:50:25.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
105962025-11-12T20:50:25.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105972025-11-12T20:50:25.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105982025-11-12T20:50:25.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
105992025-11-12T20:50:25.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.309951ms iteration = 30 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
106002025-11-12T20:50:27.940ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 319 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = c39a66ff-ba65-41d6-8dc4-0806b347dc4f response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
106012025-11-12T20:50:30.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 60 reason = Timeout
106022025-11-12T20:50:30.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106032025-11-12T20:50:30.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
106042025-11-12T20:50:30.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106052025-11-12T20:50:30.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
106062025-11-12T20:50:30.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106072025-11-12T20:50:30.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
106082025-11-12T20:50:30.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
106092025-11-12T20:50:30.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.002812ms iteration = 60 status = {"config_updated":false}
106102025-11-12T20:50:32.917ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:35636
106112025-11-12T20:50:32.917ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = f5150033-8f73-445b-95cc-9949220a338e uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
106122025-11-12T20:50:32.917ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = f5150033-8f73-445b-95cc-9949220a338e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
106132025-11-12T20:50:32.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = f5150033-8f73-445b-95cc-9949220a338e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
106142025-11-12T20:50:32.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2701 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = f5150033-8f73-445b-95cc-9949220a338e response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
106152025-11-12T20:50:32.920ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = 036ab13f-089e-4f9b-ab08-11e1a1761f31 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
106162025-11-12T20:50:32.920ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = 036ab13f-089e-4f9b-ab08-11e1a1761f31 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
106172025-11-12T20:50:32.920ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = 036ab13f-089e-4f9b-ab08-11e1a1761f31 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
106182025-11-12T20:50:32.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 1998 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35636 req_id = 036ab13f-089e-4f9b-ab08-11e1a1761f31 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
106192025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
106202025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
106212025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
106222025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
106232025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
106242025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
106252025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 11 reason = Timeout
106262025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
106272025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
106282025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
106292025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
106302025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
106312025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
106322025-11-12T20:50:35.276ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:40:35.276518114 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
106332025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
106342025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
106352025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
106362025-11-12T20:50:35.276ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
106372025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
106382025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
106392025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
106402025-11-12T20:50:35.276ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = ExternalNetworking step id = Ensure
106412025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
106422025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
106432025-11-12T20:50:35.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106442025-11-12T20:50:35.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1230, tv_nsec: 771805753 } })
106452025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
106462025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
106472025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
106482025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
106492025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
106502025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
106512025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106522025-11-12T20:50:35.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1230, tv_nsec: 772024279 } })
106532025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
106542025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.15032ms iteration = 6 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
106552025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
106562025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.341685ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
106572025-11-12T20:50:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
106582025-11-12T20:50:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
106592025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 27 reason = Timeout
106602025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
106612025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 9 reason = Timeout
106622025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
106632025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
106642025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
106652025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
106662025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
106672025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
106682025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
106692025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
106702025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
106712025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
106722025-11-12T20:50:35.279ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
106732025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106742025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
106752025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
106762025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 2 reason = Timeout
106772025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
106782025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
106792025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
106802025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
106812025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
106822025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
106832025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
106842025-11-12T20:50:35.279ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
106852025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
106862025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
106872025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
106882025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
106892025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
106902025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
106912025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
106922025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
106932025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106942025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
106952025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106962025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
106972025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
106982025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
106992025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
107002025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107012025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
107022025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
107032025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
107042025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
107052025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107062025-11-12T20:50:35.279ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
107072025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
107082025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
107092025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107102025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 61 reason = Timeout
107112025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 11 reason = Timeout
107122025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
107132025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
107142025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
107152025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
107162025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
107172025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
107182025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
107192025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
107202025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
107212025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
107222025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
107232025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
107242025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
107252025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
107262025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
107272025-11-12T20:50:35.280ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
107282025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
107292025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
107302025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
107312025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107322025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
107332025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
107342025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
107352025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
107362025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
107372025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
107382025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
107392025-11-12T20:50:35.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1230, tv_nsec: 775184831 } })
107402025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
107412025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
107422025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
107432025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
107442025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
107452025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
107462025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107472025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
107482025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
107492025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
107502025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
107512025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107522025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107532025-11-12T20:50:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
107542025-11-12T20:50:35.282ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1230, tv_nsec: 777184652 } })
107552025-11-12T20:50:35.282ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107562025-11-12T20:50:35.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107572025-11-12T20:50:35.282ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1230, tv_nsec: 777650785 } })
107582025-11-12T20:50:35.283ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
107592025-11-12T20:50:35.284ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:40:35.276518114 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
107602025-11-12T20:50:35.284ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.038468ms iteration = 6 status = {"expiration":"2025-11-12T20:40:35.276518114Z","pruned":[]}
107612025-11-12T20:50:35.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107622025-11-12T20:50:35.284ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107632025-11-12T20:50:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107642025-11-12T20:50:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107652025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107662025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107672025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107682025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 21 reason = Timeout
107692025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
107702025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107712025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
107722025-11-12T20:50:35.286ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107732025-11-12T20:50:35.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
107742025-11-12T20:50:35.287ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
107752025-11-12T20:50:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
107762025-11-12T20:50:35.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107772025-11-12T20:50:35.287ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.384245ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
107782025-11-12T20:50:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
107792025-11-12T20:50:35.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107802025-11-12T20:50:35.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
107812025-11-12T20:50:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
107822025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
107832025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
107842025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
107852025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 6 reason = Timeout
107862025-11-12T20:50:35.292ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
107872025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
107882025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 21 reason = Timeout
107892025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
107902025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
107912025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
107922025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
107932025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
107942025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
107952025-11-12T20:50:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
107962025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
107972025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
107982025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
107992025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
108002025-11-12T20:50:35.293ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
108012025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 13.831008ms iteration = 6 status = {"by_rx":{},"error":null}
108022025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
108032025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
108042025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
108052025-11-12T20:50:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
108062025-11-12T20:50:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
108072025-11-12T20:50:35.294ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
108082025-11-12T20:50:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
108092025-11-12T20:50:35.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
108102025-11-12T20:50:35.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
108112025-11-12T20:50:35.295ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 15.397369ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
108122025-11-12T20:50:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
108132025-11-12T20:50:35.296ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
108142025-11-12T20:50:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
108152025-11-12T20:50:35.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108162025-11-12T20:50:35.298ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
108172025-11-12T20:50:35.298ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
108182025-11-12T20:50:35.298ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
108192025-11-12T20:50:35.298ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
108202025-11-12T20:50:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
108212025-11-12T20:50:35.300ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108222025-11-12T20:50:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108232025-11-12T20:50:35.301ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108242025-11-12T20:50:35.301ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108252025-11-12T20:50:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
108262025-11-12T20:50:35.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108272025-11-12T20:50:35.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108282025-11-12T20:50:35.302ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108292025-11-12T20:50:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
108302025-11-12T20:50:35.303ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 22.962655ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
108312025-11-12T20:50:35.304ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108322025-11-12T20:50:35.305ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108332025-11-12T20:50:35.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108342025-11-12T20:50:35.306ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
108352025-11-12T20:50:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
108362025-11-12T20:50:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
108372025-11-12T20:50:35.307ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
108382025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
108392025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108402025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
108412025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
108422025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108432025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108442025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108452025-11-12T20:50:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108462025-11-12T20:50:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
108472025-11-12T20:50:35.309ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
108482025-11-12T20:50:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
108492025-11-12T20:50:35.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
108502025-11-12T20:50:35.310ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
108512025-11-12T20:50:35.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
108522025-11-12T20:50:35.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
108532025-11-12T20:50:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
108542025-11-12T20:50:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
108552025-11-12T20:50:35.312ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108562025-11-12T20:50:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
108572025-11-12T20:50:35.312ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
108582025-11-12T20:50:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
108592025-11-12T20:50:35.313ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
108602025-11-12T20:50:35.313ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
108612025-11-12T20:50:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
108622025-11-12T20:50:35.314ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108632025-11-12T20:50:35.314ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
108642025-11-12T20:50:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
108652025-11-12T20:50:35.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108662025-11-12T20:50:35.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
108672025-11-12T20:50:35.316ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
108682025-11-12T20:50:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108692025-11-12T20:50:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
108702025-11-12T20:50:35.316ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
108712025-11-12T20:50:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
108722025-11-12T20:50:35.317ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1230, tv_nsec: 812025294 } })
108732025-11-12T20:50:35.317ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
108742025-11-12T20:50:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108752025-11-12T20:50:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
108762025-11-12T20:50:35.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
108772025-11-12T20:50:35.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
108782025-11-12T20:50:35.318ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
108792025-11-12T20:50:35.318ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 39.333147ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
108802025-11-12T20:50:35.318ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
108812025-11-12T20:50:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
108822025-11-12T20:50:35.319ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108832025-11-12T20:50:35.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
108842025-11-12T20:50:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
108852025-11-12T20:50:35.321ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 original_time_started = 2025-11-12 20:49:36.625152 UTC
108862025-11-12T20:50:35.322ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 43.112936ms iteration = 27 status = {"Loaded":{"collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","time_loaded":"2025-11-12T20:50:35.278898036Z","time_started":"2025-11-12T20:49:36.625152Z"}}
108872025-11-12T20:50:35.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
108882025-11-12T20:50:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
108892025-11-12T20:50:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
108902025-11-12T20:50:35.323ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
108912025-11-12T20:50:35.323ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108922025-11-12T20:50:35.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
108932025-11-12T20:50:35.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108942025-11-12T20:50:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
108952025-11-12T20:50:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
108962025-11-12T20:50:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108972025-11-12T20:50:35.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
108982025-11-12T20:50:35.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 45.583879ms iteration = 61 status = {"config_updated":false}
108992025-11-12T20:50:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
109002025-11-12T20:50:35.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
109012025-11-12T20:50:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
109022025-11-12T20:50:35.326ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109032025-11-12T20:50:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
109042025-11-12T20:50:35.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
109052025-11-12T20:50:35.330ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
109062025-11-12T20:50:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
109072025-11-12T20:50:35.333ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
109082025-11-12T20:50:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
109092025-11-12T20:50:35.344ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 57.944529ms iteration = 21 status = "NoSitrep"
109102025-11-12T20:50:35.345ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
109112025-11-12T20:50:35.345ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
109122025-11-12T20:50:35.345ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
109132025-11-12T20:50:35.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
109142025-11-12T20:50:35.346ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 66.516492ms iteration = 11 status = 0
109152025-11-12T20:50:35.347ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109162025-11-12T20:50:35.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
109172025-11-12T20:50:35.349ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 69.331814ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
109182025-11-12T20:50:35.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
109192025-11-12T20:50:35.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109202025-11-12T20:50:35.351ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
109212025-11-12T20:50:35.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
109222025-11-12T20:50:35.351ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
109232025-11-12T20:50:35.351ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
109242025-11-12T20:50:35.352ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109252025-11-12T20:50:35.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
109262025-11-12T20:50:35.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109272025-11-12T20:50:35.353ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109282025-11-12T20:50:35.354ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109292025-11-12T20:50:35.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109302025-11-12T20:50:35.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109312025-11-12T20:50:35.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
109322025-11-12T20:50:35.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
109332025-11-12T20:50:35.357ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
109342025-11-12T20:50:35.357ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109352025-11-12T20:50:35.357ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109362025-11-12T20:50:35.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
109372025-11-12T20:50:35.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109382025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
109392025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
109402025-11-12T20:50:35.360ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
109412025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
109422025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109432025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
109442025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
109452025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109462025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
109472025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109482025-11-12T20:50:35.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109492025-11-12T20:50:35.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109502025-11-12T20:50:35.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
109512025-11-12T20:50:35.361ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
109522025-11-12T20:50:35.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
109532025-11-12T20:50:35.363ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
109542025-11-12T20:50:35.363ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
109552025-11-12T20:50:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
109562025-11-12T20:50:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
109572025-11-12T20:50:35.364ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
109582025-11-12T20:50:35.364ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 84.517728ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
109592025-11-12T20:50:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
109602025-11-12T20:50:35.365ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
109612025-11-12T20:50:35.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
109622025-11-12T20:50:35.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
109632025-11-12T20:50:35.370ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
109642025-11-12T20:50:35.373ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
109652025-11-12T20:50:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
109662025-11-12T20:50:35.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109672025-11-12T20:50:35.378ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109682025-11-12T20:50:35.379ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109692025-11-12T20:50:35.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109702025-11-12T20:50:35.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109712025-11-12T20:50:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109722025-11-12T20:50:35.381ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
109732025-11-12T20:50:35.382ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
109742025-11-12T20:50:35.382ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
109752025-11-12T20:50:35.383ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 90.80938ms iteration = 21 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
109762025-11-12T20:50:35.385ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
109772025-11-12T20:50:35.385ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 105.50898ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
109782025-11-12T20:50:35.386ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
109792025-11-12T20:50:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
109802025-11-12T20:50:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
109812025-11-12T20:50:35.387ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109822025-11-12T20:50:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109832025-11-12T20:50:35.392ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
109842025-11-12T20:50:35.392ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
109852025-11-12T20:50:35.392ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
109862025-11-12T20:50:35.392ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 115.575311ms iteration = 6 status = {"generation":2}
109872025-11-12T20:50:35.393ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109882025-11-12T20:50:35.393ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
109892025-11-12T20:50:35.395ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
109902025-11-12T20:50:35.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109912025-11-12T20:50:35.395ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109922025-11-12T20:50:35.396ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
109932025-11-12T20:50:35.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
109942025-11-12T20:50:35.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109952025-11-12T20:50:35.397ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
109962025-11-12T20:50:35.398ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
109972025-11-12T20:50:35.398ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109982025-11-12T20:50:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
109992025-11-12T20:50:35.398ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 119.361549ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
110002025-11-12T20:50:35.399ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
110012025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
110022025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
110032025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
110042025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110052025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
110062025-11-12T20:50:35.400ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
110072025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 121.462483ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
110082025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
110092025-11-12T20:50:35.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110102025-11-12T20:50:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110112025-11-12T20:50:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110122025-11-12T20:50:35.401ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
110132025-11-12T20:50:35.402ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
110142025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110152025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110162025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110172025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
110182025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
110192025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
110202025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 123.472825ms iteration = 31 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
110212025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110222025-11-12T20:50:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
110232025-11-12T20:50:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
110242025-11-12T20:50:35.404ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
110252025-11-12T20:50:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
110262025-11-12T20:50:35.404ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
110272025-11-12T20:50:35.405ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = SledList step id = Fetch
110282025-11-12T20:50:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
110292025-11-12T20:50:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
110302025-11-12T20:50:35.406ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
110312025-11-12T20:50:35.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110322025-11-12T20:50:35.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110332025-11-12T20:50:35.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110342025-11-12T20:50:35.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110352025-11-12T20:50:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
110362025-11-12T20:50:35.411ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110372025-11-12T20:50:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110382025-11-12T20:50:35.412ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
110392025-11-12T20:50:35.412ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
110402025-11-12T20:50:35.412ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
110412025-11-12T20:50:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
110422025-11-12T20:50:35.413ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:114 status = TufRepoPrunerStatus { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
110432025-11-12T20:50:35.413ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 134.265524ms iteration = 2 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"warnings":[]}
110442025-11-12T20:50:35.416ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
110452025-11-12T20:50:35.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
110462025-11-12T20:50:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
110472025-11-12T20:50:35.416ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110482025-11-12T20:50:35.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110492025-11-12T20:50:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110502025-11-12T20:50:35.418ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
110512025-11-12T20:50:35.418ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
110522025-11-12T20:50:35.418ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
110532025-11-12T20:50:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
110542025-11-12T20:50:35.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110552025-11-12T20:50:35.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110562025-11-12T20:50:35.425ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110572025-11-12T20:50:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110582025-11-12T20:50:35.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
110592025-11-12T20:50:35.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
110602025-11-12T20:50:35.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110612025-11-12T20:50:35.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110622025-11-12T20:50:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
110632025-11-12T20:50:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110642025-11-12T20:50:35.431ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110652025-11-12T20:50:35.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110662025-11-12T20:50:35.432ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
110672025-11-12T20:50:35.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
110682025-11-12T20:50:35.435ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110692025-11-12T20:50:35.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110702025-11-12T20:50:35.437ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = DeployNexusRecords step id = Ensure
110712025-11-12T20:50:35.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
110722025-11-12T20:50:35.437ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 158.001359ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
110732025-11-12T20:50:35.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
110742025-11-12T20:50:35.439ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
110752025-11-12T20:50:35.439ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 160.087463ms iteration = 9 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"0f5ca54a-65d9-456c-b99e-32ccac684146","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
110762025-11-12T20:50:35.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110772025-11-12T20:50:35.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110782025-11-12T20:50:35.444ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
110792025-11-12T20:50:35.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
110802025-11-12T20:50:35.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
110812025-11-12T20:50:35.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
110822025-11-12T20:50:35.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
110832025-11-12T20:50:35.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
110842025-11-12T20:50:35.448ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110852025-11-12T20:50:35.449ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
110862025-11-12T20:50:35.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110872025-11-12T20:50:35.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
110882025-11-12T20:50:35.452ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110892025-11-12T20:50:35.453ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110902025-11-12T20:50:35.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110912025-11-12T20:50:35.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110922025-11-12T20:50:35.456ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110932025-11-12T20:50:35.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
110942025-11-12T20:50:35.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
110952025-11-12T20:50:35.457ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
110962025-11-12T20:50:35.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
110972025-11-12T20:50:35.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
110982025-11-12T20:50:35.459ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
110992025-11-12T20:50:35.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111002025-11-12T20:50:35.462ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111012025-11-12T20:50:35.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
111022025-11-12T20:50:35.470ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111032025-11-12T20:50:35.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111042025-11-12T20:50:35.473ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111052025-11-12T20:50:35.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111062025-11-12T20:50:35.477ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111072025-11-12T20:50:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111082025-11-12T20:50:35.479ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
111092025-11-12T20:50:35.479ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
111102025-11-12T20:50:35.479ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 199.450762ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
111112025-11-12T20:50:35.481ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 201.276078ms iteration = 18 status = {"physical_disks_added":0}
111122025-11-12T20:50:35.482ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111132025-11-12T20:50:35.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111142025-11-12T20:50:35.485ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111152025-11-12T20:50:35.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111162025-11-12T20:50:35.493ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111172025-11-12T20:50:35.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111182025-11-12T20:50:35.498ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111192025-11-12T20:50:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111202025-11-12T20:50:35.501ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111212025-11-12T20:50:35.504ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
111222025-11-12T20:50:35.504ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 228.394371ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
111232025-11-12T20:50:35.505ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 225.487285ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
111242025-11-12T20:50:35.509ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
111252025-11-12T20:50:35.509ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111262025-11-12T20:50:35.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111272025-11-12T20:50:35.515ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111282025-11-12T20:50:35.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111292025-11-12T20:50:35.520ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111302025-11-12T20:50:35.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 240.619017ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
111312025-11-12T20:50:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111322025-11-12T20:50:35.522ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111332025-11-12T20:50:35.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111342025-11-12T20:50:35.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111352025-11-12T20:50:35.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111362025-11-12T20:50:35.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111372025-11-12T20:50:35.529ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
111382025-11-12T20:50:35.529ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
111392025-11-12T20:50:35.529ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 249.695901ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
111402025-11-12T20:50:35.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
111412025-11-12T20:50:35.530ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111422025-11-12T20:50:35.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111432025-11-12T20:50:35.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111442025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111452025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111462025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111472025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
111482025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
111492025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
111502025-11-12T20:50:35.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
111512025-11-12T20:50:35.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
111522025-11-12T20:50:35.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111532025-11-12T20:50:35.539ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111542025-11-12T20:50:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111552025-11-12T20:50:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111562025-11-12T20:50:35.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111572025-11-12T20:50:35.540ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = SledAgent step id = Ensure
111582025-11-12T20:50:35.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111592025-11-12T20:50:35.542ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111602025-11-12T20:50:35.542ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
111612025-11-12T20:50:35.542ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 262.909024ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
111622025-11-12T20:50:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111632025-11-12T20:50:35.545ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111642025-11-12T20:50:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111652025-11-12T20:50:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
111662025-11-12T20:50:35.546ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
111672025-11-12T20:50:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
111682025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
111692025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111702025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
111712025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
111722025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
111732025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111742025-11-12T20:50:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
111752025-11-12T20:50:35.550ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111762025-11-12T20:50:35.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111772025-11-12T20:50:35.550ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111782025-11-12T20:50:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111792025-11-12T20:50:35.555ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111802025-11-12T20:50:35.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111812025-11-12T20:50:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
111822025-11-12T20:50:35.555ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
111832025-11-12T20:50:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111842025-11-12T20:50:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
111852025-11-12T20:50:35.556ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111862025-11-12T20:50:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111872025-11-12T20:50:35.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111882025-11-12T20:50:35.558ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111892025-11-12T20:50:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111902025-11-12T20:50:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111912025-11-12T20:50:35.560ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
111922025-11-12T20:50:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
111932025-11-12T20:50:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111942025-11-12T20:50:35.561ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
111952025-11-12T20:50:35.561ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
111962025-11-12T20:50:35.561ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
111972025-11-12T20:50:35.561ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 285.312804ms iteration = 6 status = {"generation":1}
111982025-11-12T20:50:35.563ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111992025-11-12T20:50:35.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
112002025-11-12T20:50:35.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112012025-11-12T20:50:35.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112022025-11-12T20:50:35.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112032025-11-12T20:50:35.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112042025-11-12T20:50:35.571ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
112052025-11-12T20:50:35.571ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112062025-11-12T20:50:35.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112072025-11-12T20:50:35.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112082025-11-12T20:50:35.573ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112092025-11-12T20:50:35.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112102025-11-12T20:50:35.576ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112112025-11-12T20:50:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112122025-11-12T20:50:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
112132025-11-12T20:50:35.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112142025-11-12T20:50:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112152025-11-12T20:50:35.580ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112162025-11-12T20:50:35.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112172025-11-12T20:50:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112182025-11-12T20:50:35.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112192025-11-12T20:50:35.588ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
112202025-11-12T20:50:35.588ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112212025-11-12T20:50:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112222025-11-12T20:50:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
112232025-11-12T20:50:35.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112242025-11-12T20:50:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112252025-11-12T20:50:35.605ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
112262025-11-12T20:50:35.605ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112272025-11-12T20:50:35.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112282025-11-12T20:50:35.608ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112292025-11-12T20:50:35.612ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
112302025-11-12T20:50:35.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
112312025-11-12T20:50:35.613ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
112322025-11-12T20:50:35.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
112332025-11-12T20:50:35.615ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
112342025-11-12T20:50:35.615ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
112352025-11-12T20:50:35.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
112362025-11-12T20:50:35.618ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
112372025-11-12T20:50:35.618ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
112382025-11-12T20:50:35.624ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
112392025-11-12T20:50:35.625ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a813945-8a8d-42a1-9584-884a682cb212", "content-length": "78", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
112402025-11-12T20:50:35.625ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
112412025-11-12T20:50:35.625ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
112422025-11-12T20:50:35.625ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
112432025-11-12T20:50:35.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
112442025-11-12T20:50:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
112452025-11-12T20:50:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
112462025-11-12T20:50:35.628ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
112472025-11-12T20:50:35.632ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
112482025-11-12T20:50:35.632ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
112492025-11-12T20:50:35.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
112502025-11-12T20:50:35.635ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
112512025-11-12T20:50:35.636ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
112522025-11-12T20:50:35.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
112532025-11-12T20:50:35.640ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
112542025-11-12T20:50:35.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
112552025-11-12T20:50:35.640ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
112562025-11-12T20:50:35.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
112572025-11-12T20:50:35.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
112582025-11-12T20:50:35.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112592025-11-12T20:50:35.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112602025-11-12T20:50:35.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112612025-11-12T20:50:35.650ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
112622025-11-12T20:50:35.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112632025-11-12T20:50:35.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112642025-11-12T20:50:35.656ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112652025-11-12T20:50:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
112662025-11-12T20:50:35.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
112672025-11-12T20:50:35.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112682025-11-12T20:50:35.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112692025-11-12T20:50:35.697ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
112702025-11-12T20:50:35.698ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
112712025-11-12T20:50:35.698ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cf473ab-e25a-4a2f-8747-897953c57bf4", "content-length": "81", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
112722025-11-12T20:50:35.708ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
112732025-11-12T20:50:35.708ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
112742025-11-12T20:50:35.709ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "af2247db-6725-4f36-9490-e3b2bb7ca65c", "content-length": "81", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
112752025-11-12T20:50:35.709ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
112762025-11-12T20:50:35.709ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
112772025-11-12T20:50:35.710ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "55f45475-e83f-4c79-a01a-2b17aa7cc66f", "content-length": "81", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
112782025-11-12T20:50:35.711ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 431.927157ms iteration = 11 status = {"disabled":false,"errors":[],"sps":[]}
112792025-11-12T20:50:35.714ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
112802025-11-12T20:50:35.714ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112812025-11-12T20:50:35.715ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1989a5e-c37f-4dc3-a595-4fe20f8e47f1", "content-length": "26", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
112822025-11-12T20:50:35.715ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112832025-11-12T20:50:35.718ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
112842025-11-12T20:50:35.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112852025-11-12T20:50:35.725ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112862025-11-12T20:50:35.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
112872025-11-12T20:50:35.729ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112882025-11-12T20:50:35.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
112892025-11-12T20:50:35.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112902025-11-12T20:50:35.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
112912025-11-12T20:50:35.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112922025-11-12T20:50:35.739ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
112932025-11-12T20:50:35.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112942025-11-12T20:50:35.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112952025-11-12T20:50:35.741ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 464.703626ms iteration = 11 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
112962025-11-12T20:50:35.741ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
112972025-11-12T20:50:35.742ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
112982025-11-12T20:50:35.742ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e15d899a-7ff3-4bcb-b725-6e6252196253", "content-length": "372", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
112992025-11-12T20:50:35.742ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113002025-11-12T20:50:35.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113012025-11-12T20:50:35.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113022025-11-12T20:50:35.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113032025-11-12T20:50:35.745ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
113042025-11-12T20:50:35.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113052025-11-12T20:50:35.746ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "28ef7436-925c-4749-8369-23ea45e685d5", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
113062025-11-12T20:50:35.746ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
113072025-11-12T20:50:35.748ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
113082025-11-12T20:50:35.748ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 456.217007ms iteration = 6 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
113092025-11-12T20:50:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
113102025-11-12T20:50:35.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113112025-11-12T20:50:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113122025-11-12T20:50:35.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113132025-11-12T20:50:35.755ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
113142025-11-12T20:50:35.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113152025-11-12T20:50:35.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113162025-11-12T20:50:35.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113172025-11-12T20:50:35.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113182025-11-12T20:50:35.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113192025-11-12T20:50:35.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113202025-11-12T20:50:35.766ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
113212025-11-12T20:50:35.766ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
113222025-11-12T20:50:35.766ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe5091e-e980-443d-97b9-d016639c2b21", "content-length": "26", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
113232025-11-12T20:50:35.766ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
113242025-11-12T20:50:35.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113252025-11-12T20:50:35.771ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
113262025-11-12T20:50:35.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
113272025-11-12T20:50:35.792ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113282025-11-12T20:50:35.795ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
113292025-11-12T20:50:35.795ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113302025-11-12T20:50:35.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
113312025-11-12T20:50:35.800ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
113322025-11-12T20:50:35.800ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 521.116286ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
113332025-11-12T20:50:35.803ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113342025-11-12T20:50:35.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113352025-11-12T20:50:35.805ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113362025-11-12T20:50:35.809ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
113372025-11-12T20:50:35.809ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113382025-11-12T20:50:35.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113392025-11-12T20:50:35.812ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113402025-11-12T20:50:35.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113412025-11-12T20:50:35.813ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
113422025-11-12T20:50:35.814ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113432025-11-12T20:50:35.815ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "f93d4020-4ccd-4012-b85a-11f854fa7d71", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
113442025-11-12T20:50:35.816ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = FirewallRules step id = Ensure
113452025-11-12T20:50:35.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
113462025-11-12T20:50:35.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
113472025-11-12T20:50:35.816ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113482025-11-12T20:50:35.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113492025-11-12T20:50:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
113502025-11-12T20:50:35.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
113512025-11-12T20:50:35.819ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113522025-11-12T20:50:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
113532025-11-12T20:50:35.819ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
113542025-11-12T20:50:35.819ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
113552025-11-12T20:50:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113562025-11-12T20:50:35.819ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "394aaf1c-692b-4cc8-a78f-d2aeea5d3e83", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
113572025-11-12T20:50:35.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113582025-11-12T20:50:35.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113592025-11-12T20:50:35.821ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113602025-11-12T20:50:35.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
113612025-11-12T20:50:35.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
113622025-11-12T20:50:35.822ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113632025-11-12T20:50:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113642025-11-12T20:50:35.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
113652025-11-12T20:50:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
113662025-11-12T20:50:35.824ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
113672025-11-12T20:50:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
113682025-11-12T20:50:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
113692025-11-12T20:50:35.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
113702025-11-12T20:50:35.825ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113712025-11-12T20:50:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113722025-11-12T20:50:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
113732025-11-12T20:50:35.829ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
113742025-11-12T20:50:35.829ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
113752025-11-12T20:50:35.830ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "67db7eb0-c48a-4be5-bf49-d45ebaa7ff32", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
113762025-11-12T20:50:35.831ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 551.847491ms iteration = 16 status = {}
113772025-11-12T20:50:35.831ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Dependency
113782025-11-12T20:50:35.832ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
113792025-11-12T20:50:35.832ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113802025-11-12T20:50:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113812025-11-12T20:50:35.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113822025-11-12T20:50:35.833ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113832025-11-12T20:50:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113842025-11-12T20:50:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113852025-11-12T20:50:35.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113862025-11-12T20:50:35.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
113872025-11-12T20:50:35.835ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113882025-11-12T20:50:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113892025-11-12T20:50:35.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113902025-11-12T20:50:35.836ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
113912025-11-12T20:50:35.836ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
113922025-11-12T20:50:35.837ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113932025-11-12T20:50:35.838ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
113942025-11-12T20:50:35.838ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "62e85cac-8d0b-48f8-b6f6-3da2bea8e16d", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
113952025-11-12T20:50:35.838ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
113962025-11-12T20:50:35.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
113972025-11-12T20:50:35.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
113982025-11-12T20:50:35.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113992025-11-12T20:50:35.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114002025-11-12T20:50:35.841ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f692556c-b9c9-4d4e-bc5f-39b71b7a92a0", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
114012025-11-12T20:50:35.841ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
114022025-11-12T20:50:35.841ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
114032025-11-12T20:50:35.842ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "652559f8-b674-4625-9004-bd4671d72f96", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
114042025-11-12T20:50:35.842ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
114052025-11-12T20:50:35.842ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "dae823a3-85ad-45ce-9a00-5ee3657599d1", "content-length": "26", "date": "Wed, 12 Nov 2025 20:50:35 GMT"} })
114062025-11-12T20:50:35.842ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
114072025-11-12T20:50:35.849ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 569.644542ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"cdf575e5-7c27-4aa3-b1ec-77468da13285","time":"2025-11-12T20:50:35.842248705Z"},{"operation":"list","target_sled":"cdf575e5-7c27-4aa3-b1ec-77468da13285","time":"2025-11-12T20:50:35.842479851Z"}]}
114082025-11-12T20:50:35.850ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
114092025-11-12T20:50:35.850ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 573.809159ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
114102025-11-12T20:50:35.853ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
114112025-11-12T20:50:35.873ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 368408192 } })
114122025-11-12T20:50:35.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114132025-11-12T20:50:35.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114142025-11-12T20:50:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114152025-11-12T20:50:35.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
114162025-11-12T20:50:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114172025-11-12T20:50:35.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114182025-11-12T20:50:35.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114192025-11-12T20:50:35.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114202025-11-12T20:50:35.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114212025-11-12T20:50:35.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114222025-11-12T20:50:35.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114232025-11-12T20:50:35.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114242025-11-12T20:50:35.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114252025-11-12T20:50:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
114262025-11-12T20:50:35.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114272025-11-12T20:50:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114282025-11-12T20:50:35.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114292025-11-12T20:50:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114302025-11-12T20:50:35.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114312025-11-12T20:50:35.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114322025-11-12T20:50:35.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114332025-11-12T20:50:35.956ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
114342025-11-12T20:50:35.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114352025-11-12T20:50:35.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114362025-11-12T20:50:35.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114372025-11-12T20:50:35.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114382025-11-12T20:50:35.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114392025-11-12T20:50:35.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
114402025-11-12T20:50:35.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114412025-11-12T20:50:35.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114422025-11-12T20:50:35.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114432025-11-12T20:50:35.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114442025-11-12T20:50:35.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114452025-11-12T20:50:35.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
114462025-11-12T20:50:35.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114472025-11-12T20:50:35.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114482025-11-12T20:50:35.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114492025-11-12T20:50:35.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114502025-11-12T20:50:35.993ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114512025-11-12T20:50:35.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114522025-11-12T20:50:35.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114532025-11-12T20:50:35.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114542025-11-12T20:50:35.995ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 719.030407ms iteration = 11 status = {}
114552025-11-12T20:50:35.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114562025-11-12T20:50:35.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114572025-11-12T20:50:35.996ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
114582025-11-12T20:50:35.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
114592025-11-12T20:50:35.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114602025-11-12T20:50:35.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114612025-11-12T20:50:35.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114622025-11-12T20:50:35.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114632025-11-12T20:50:36.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114642025-11-12T20:50:36.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114652025-11-12T20:50:36.010ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
114662025-11-12T20:50:36.011ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "aa63101f-2f5d-449d-adc2-8691998a0d0f", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114672025-11-12T20:50:36.011ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114682025-11-12T20:50:36.013ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "74bdafcd-6605-4b9d-90a4-a576975ca316", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114692025-11-12T20:50:36.013ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114702025-11-12T20:50:36.014ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
114712025-11-12T20:50:36.014ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
114722025-11-12T20:50:36.015ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "94c1669d-db43-4dc1-8da2-cab3a3eabbff", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114732025-11-12T20:50:36.015ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114742025-11-12T20:50:36.015ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c94b117f-dac6-4d3c-aad5-adf361e8a2e3", "content-length": "372", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
114752025-11-12T20:50:36.016ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114762025-11-12T20:50:36.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114772025-11-12T20:50:36.018ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
114782025-11-12T20:50:36.018ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 741.651562ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
114792025-11-12T20:50:36.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114802025-11-12T20:50:36.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
114812025-11-12T20:50:36.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114822025-11-12T20:50:36.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114832025-11-12T20:50:36.032ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114842025-11-12T20:50:36.032ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
114852025-11-12T20:50:36.033ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "319ae6c0-9849-4e32-9dfc-1d4b045f2465", "content-length": "691", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
114862025-11-12T20:50:36.033ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
114872025-11-12T20:50:36.033ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
114882025-11-12T20:50:36.033ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
114892025-11-12T20:50:36.036ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
114902025-11-12T20:50:36.036ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 756.648471ms iteration = 11 status = {}
114912025-11-12T20:50:36.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114922025-11-12T20:50:36.046ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
114932025-11-12T20:50:36.049ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c671a0b5-41e2-4165-aa72-e559aa07a856", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
114942025-11-12T20:50:36.050ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
114952025-11-12T20:50:36.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114962025-11-12T20:50:36.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114972025-11-12T20:50:36.051ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
114982025-11-12T20:50:36.051ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 771.967047ms
114992025-11-12T20:50:36.051ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 771.976747ms iteration = 2 status = {}
115002025-11-12T20:50:36.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115012025-11-12T20:50:36.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115022025-11-12T20:50:36.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115032025-11-12T20:50:36.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115042025-11-12T20:50:36.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115052025-11-12T20:50:36.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115062025-11-12T20:50:36.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115072025-11-12T20:50:36.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115082025-11-12T20:50:36.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115092025-11-12T20:50:36.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115102025-11-12T20:50:36.064ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
115112025-11-12T20:50:36.064ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
115122025-11-12T20:50:36.064ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 2a2930ac-d3e6-478e-b357-8664baf35beb parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
115132025-11-12T20:50:36.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115142025-11-12T20:50:36.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115152025-11-12T20:50:36.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115162025-11-12T20:50:36.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115172025-11-12T20:50:36.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115182025-11-12T20:50:36.066ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
115192025-11-12T20:50:36.066ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
115202025-11-12T20:50:36.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115212025-11-12T20:50:36.067ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f2820f7b-4b66-4739-b6de-94a877477a14", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
115222025-11-12T20:50:36.067ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115232025-11-12T20:50:36.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115242025-11-12T20:50:36.069ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115252025-11-12T20:50:36.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115262025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
115272025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115282025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
115292025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115302025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115312025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115322025-11-12T20:50:36.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115332025-11-12T20:50:36.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115342025-11-12T20:50:36.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115352025-11-12T20:50:36.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115362025-11-12T20:50:36.074ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
115372025-11-12T20:50:36.074ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
115382025-11-12T20:50:36.075ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 795.6515ms iteration = 15 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
115392025-11-12T20:50:36.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115402025-11-12T20:50:36.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115412025-11-12T20:50:36.076ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
115422025-11-12T20:50:36.076ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
115432025-11-12T20:50:36.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115442025-11-12T20:50:36.077ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115452025-11-12T20:50:36.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115462025-11-12T20:50:36.080ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
115472025-11-12T20:50:36.080ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115482025-11-12T20:50:36.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115492025-11-12T20:50:36.083ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
115502025-11-12T20:50:36.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115512025-11-12T20:50:36.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115522025-11-12T20:50:36.084ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f10a5de1-83c0-4e6a-8a38-2b5a1cfaa5fe", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
115532025-11-12T20:50:36.086ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 254.524946ms iteration = 17 status = {}
115542025-11-12T20:50:36.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115552025-11-12T20:50:36.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115562025-11-12T20:50:36.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115572025-11-12T20:50:36.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115582025-11-12T20:50:36.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115592025-11-12T20:50:36.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115602025-11-12T20:50:36.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115612025-11-12T20:50:36.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115622025-11-12T20:50:36.100ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
115632025-11-12T20:50:36.100ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
115642025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
115652025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
115662025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115672025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115682025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115692025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
115702025-11-12T20:50:36.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115712025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
115722025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
115732025-11-12T20:50:36.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115742025-11-12T20:50:36.101ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
115752025-11-12T20:50:36.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115762025-11-12T20:50:36.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115772025-11-12T20:50:36.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115782025-11-12T20:50:36.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115792025-11-12T20:50:36.103ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
115802025-11-12T20:50:36.103ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
115812025-11-12T20:50:36.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115822025-11-12T20:50:36.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115832025-11-12T20:50:36.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115842025-11-12T20:50:36.114ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
115852025-11-12T20:50:36.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115862025-11-12T20:50:36.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115872025-11-12T20:50:36.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115882025-11-12T20:50:36.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115892025-11-12T20:50:36.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115902025-11-12T20:50:36.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115912025-11-12T20:50:36.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115922025-11-12T20:50:36.126ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
115932025-11-12T20:50:36.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115942025-11-12T20:50:36.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115952025-11-12T20:50:36.132ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
115962025-11-12T20:50:36.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115972025-11-12T20:50:36.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115982025-11-12T20:50:36.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115992025-11-12T20:50:36.136ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
116002025-11-12T20:50:36.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116012025-11-12T20:50:36.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116022025-11-12T20:50:36.137ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
116032025-11-12T20:50:36.137ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
116042025-11-12T20:50:36.137ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
116052025-11-12T20:50:36.229ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "cf96b2fd-9feb-48b3-872c-1931928e78ac", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
116062025-11-12T20:50:36.229ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
116072025-11-12T20:50:36.229ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
116082025-11-12T20:50:36.234ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "169e36d2-0b62-4c35-b9eb-1f335865a75d", "content-length": "682", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
116092025-11-12T20:50:36.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116102025-11-12T20:50:36.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116112025-11-12T20:50:36.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116122025-11-12T20:50:36.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116132025-11-12T20:50:36.242ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116142025-11-12T20:50:36.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116152025-11-12T20:50:36.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116162025-11-12T20:50:36.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116172025-11-12T20:50:36.251ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 971.689755ms iteration = 11 status = {}
116182025-11-12T20:50:36.268ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
116192025-11-12T20:50:36.268ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
116202025-11-12T20:50:36.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116212025-11-12T20:50:36.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116222025-11-12T20:50:36.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116232025-11-12T20:50:36.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116242025-11-12T20:50:36.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116252025-11-12T20:50:36.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
116262025-11-12T20:50:36.310ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
116272025-11-12T20:50:36.311ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ee1fc932-9d06-40f1-8063-f828f08702ba", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
116282025-11-12T20:50:36.313ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
116292025-11-12T20:50:36.313ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = Dns step id = Ensure
116302025-11-12T20:50:36.313ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
116312025-11-12T20:50:36.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116322025-11-12T20:50:36.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116332025-11-12T20:50:36.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116342025-11-12T20:50:36.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116352025-11-12T20:50:36.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116362025-11-12T20:50:36.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116372025-11-12T20:50:36.365ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
116382025-11-12T20:50:36.365ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
116392025-11-12T20:50:36.365ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
116402025-11-12T20:50:36.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116412025-11-12T20:50:36.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116422025-11-12T20:50:36.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116432025-11-12T20:50:36.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116442025-11-12T20:50:36.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116452025-11-12T20:50:36.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116462025-11-12T20:50:36.411ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
116472025-11-12T20:50:36.411ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
116482025-11-12T20:50:36.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116492025-11-12T20:50:36.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116502025-11-12T20:50:36.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116512025-11-12T20:50:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116522025-11-12T20:50:36.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116532025-11-12T20:50:36.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116542025-11-12T20:50:36.419ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
116552025-11-12T20:50:36.419ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
116562025-11-12T20:50:36.419ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
116572025-11-12T20:50:36.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116582025-11-12T20:50:36.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116592025-11-12T20:50:36.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116602025-11-12T20:50:36.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116612025-11-12T20:50:36.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116622025-11-12T20:50:36.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116632025-11-12T20:50:36.459ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
116642025-11-12T20:50:36.459ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
116652025-11-12T20:50:36.459ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
116662025-11-12T20:50:36.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116672025-11-12T20:50:36.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116682025-11-12T20:50:36.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116692025-11-12T20:50:36.466ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116702025-11-12T20:50:36.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116712025-11-12T20:50:36.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116722025-11-12T20:50:36.505ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
116732025-11-12T20:50:36.505ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
116742025-11-12T20:50:36.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116752025-11-12T20:50:36.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116762025-11-12T20:50:36.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116772025-11-12T20:50:36.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116782025-11-12T20:50:36.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116792025-11-12T20:50:36.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116802025-11-12T20:50:36.513ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
116812025-11-12T20:50:36.513ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
116822025-11-12T20:50:36.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116832025-11-12T20:50:36.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116842025-11-12T20:50:36.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116852025-11-12T20:50:36.517ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
116862025-11-12T20:50:36.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116872025-11-12T20:50:36.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116882025-11-12T20:50:36.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116892025-11-12T20:50:36.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116902025-11-12T20:50:36.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116912025-11-12T20:50:36.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116922025-11-12T20:50:36.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116932025-11-12T20:50:36.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
116942025-11-12T20:50:36.528ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
116952025-11-12T20:50:36.528ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
116962025-11-12T20:50:36.528ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = OmicronZones step id = Cleanup
116972025-11-12T20:50:36.528ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = OmicronZones step id = Cleanup
116982025-11-12T20:50:36.528ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = PhysicalDisks step id = Cleanup
116992025-11-12T20:50:36.528ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = Clickhouse step id = Ensure
117002025-11-12T20:50:36.528ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = Clickhouse step id = Ensure
117012025-11-12T20:50:36.562ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
117022025-11-12T20:50:36.567ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "404d16b6-baad-45f4-9c43-fbf606f89cf1", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117032025-11-12T20:50:36.568ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = SupportBundles step id = Cleanup
117042025-11-12T20:50:36.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117052025-11-12T20:50:36.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117062025-11-12T20:50:36.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117072025-11-12T20:50:36.572ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
117082025-11-12T20:50:36.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117092025-11-12T20:50:36.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117102025-11-12T20:50:36.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117112025-11-12T20:50:36.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117122025-11-12T20:50:36.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117132025-11-12T20:50:36.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
117142025-11-12T20:50:36.633ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = OmicronZones step id = Cleanup
117152025-11-12T20:50:36.633ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
117162025-11-12T20:50:36.633ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
117172025-11-12T20:50:36.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117182025-11-12T20:50:36.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117192025-11-12T20:50:36.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117202025-11-12T20:50:36.638ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
117212025-11-12T20:50:36.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117222025-11-12T20:50:36.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117232025-11-12T20:50:36.689ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
117242025-11-12T20:50:36.689ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
117252025-11-12T20:50:36.689ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = Cockroach step id = Ensure
117262025-11-12T20:50:36.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117272025-11-12T20:50:36.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117282025-11-12T20:50:36.717ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
117292025-11-12T20:50:36.717ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a99f4cfc-1155-4a52-8474-7fcfa9e609b0 step component = MgsUpdates step id = Ensure
117302025-11-12T20:50:36.717ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Dependency
117312025-11-12T20:50:36.717ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Dependency
117322025-11-12T20:50:36.717ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
117332025-11-12T20:50:36.717ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117342025-11-12T20:50:36.718ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.441492584s iteration = 7 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3871,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":129065620,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":129065560,"secs":0}},"event_index":1,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":129079471,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":31875225,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":31875175,"secs":0}},"event_index":2,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":160971906,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":103002245,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":103002205,"secs":0}},"event_index":3,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":263988932,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":276175307,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":276175257,"secs":0}},"event_index":4,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":540187910,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":497006481,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":497006441,"secs":0}},"event_index":5,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":37208432,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":214831949,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":214831920,"secs":0}},"event_index":6,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":252052981,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4270,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4230,"secs":0}},"event_index":7,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":252068752,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2240,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2200,"secs":0}},"event_index":8,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":252073052,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2900,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2860,"secs":0}},"event_index":9,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":252077502,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1760,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1730,"secs":0}},"event_index":10,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":252081072,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40176360,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":40176330,"secs":0}},"event_index":11,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":292258542,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":64272543,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":64272514,"secs":0}},"event_index":12,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":356541536,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":56476461,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":56476421,"secs":0}},"event_index":13,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":413039307,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":27940293,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":27940253,"secs":0}},"event_index":14,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":441007381,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5900,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5860,"secs":0}},"event_index":15,"execution_id":"4bbf8a6a-e248-4ff6-bc00-e3d15cb323f9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":441021352,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
117352025-11-12T20:50:36.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
117362025-11-12T20:50:36.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
117372025-11-12T20:50:36.720ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117382025-11-12T20:50:36.720ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117392025-11-12T20:50:36.724ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
117402025-11-12T20:50:36.724ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117412025-11-12T20:50:36.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
117422025-11-12T20:50:36.724ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
117432025-11-12T20:50:36.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
117442025-11-12T20:50:36.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
117452025-11-12T20:50:36.770ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(576bf650-7b8a-4a5d-bb74-2e41dc1b85ff, 0), (f4626f35-c30e-499c-b2d2-25f8187ff08d, 0), (0f5ca54a-65d9-456c-b99e-32ccac684146, 0)]
117462025-11-12T20:50:36.770ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
117472025-11-12T20:50:36.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
117482025-11-12T20:50:36.771ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1232, tv_nsec: 265823836 } })
117492025-11-12T20:50:36.772ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117502025-11-12T20:50:36.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
117512025-11-12T20:50:36.775ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
117522025-11-12T20:50:36.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
117532025-11-12T20:50:36.781ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
117542025-11-12T20:50:36.781ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 64.171001ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
117552025-11-12T20:50:36.817ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1232, tv_nsec: 312766481 } })
117562025-11-12T20:50:36.818ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
117572025-11-12T20:50:36.818ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
117582025-11-12T20:50:36.818ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
117592025-11-12T20:50:36.819ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "600a7964-0121-4e88-8743-6350f7ead849", "content-length": "476", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117602025-11-12T20:50:36.819ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
117612025-11-12T20:50:36.819ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3eff09a-3c73-40fa-9ad1-9cc0595ca6a7", "content-length": "734", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117622025-11-12T20:50:36.819ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
117632025-11-12T20:50:36.820ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "20f2a438-4742-4151-a836-55eaa2ebc966", "content-length": "111", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117642025-11-12T20:50:36.820ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
117652025-11-12T20:50:36.820ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "503a2ab6-ca20-4899-bc6b-0624850b2cb0", "content-length": "111", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117662025-11-12T20:50:36.820ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
117672025-11-12T20:50:36.821ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b1d9799-1fed-49fb-bcc7-bf1d15cce04a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117682025-11-12T20:50:36.821ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
117692025-11-12T20:50:36.821ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ba39936-668e-4b23-b66a-0daddef9cad7", "content-length": "163", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117702025-11-12T20:50:36.821ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
117712025-11-12T20:50:36.822ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ed5509-b298-40f1-bf11-799398378be2", "content-length": "166", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117722025-11-12T20:50:36.822ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
117732025-11-12T20:50:36.822ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f4c954f-769c-43fe-aac5-2800407761a0", "content-length": "166", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117742025-11-12T20:50:36.822ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
117752025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e59f580-d16c-4265-8bf8-b9032421b733", "content-length": "702", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117762025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
117772025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27387a48-1948-4f84-ad0d-d04124a796f9", "content-length": "727", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117782025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
117792025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d43d3ca-1c4a-4072-8821-fb1bf21e8ef7", "content-length": "729", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117802025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
117812025-11-12T20:50:36.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db7e760a-b239-489b-b12c-1e6ed8120889", "content-length": "728", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117822025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
117832025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21a49d43-e88b-486d-92f0-46b9b2133403", "content-length": "722", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117842025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
117852025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe14183-c445-44d4-b2bd-dc944a796419", "content-length": "10", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117862025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
117872025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec701279-50e8-4b67-b4ee-a56c21e2445a", "content-length": "144", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117882025-11-12T20:50:36.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
117892025-11-12T20:50:36.825ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "60e7870b-c4f6-4221-9076-e8c9478bc38f", "content-length": "147", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117902025-11-12T20:50:36.825ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
117912025-11-12T20:50:36.825ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cef04b94-458c-4aa3-8147-e6c3020574d9", "content-length": "109", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117922025-11-12T20:50:36.825ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
117932025-11-12T20:50:36.826ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d687cd7-5d95-4fde-9604-6442ff04f51e", "content-length": "109", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117942025-11-12T20:50:36.826ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
117952025-11-12T20:50:36.827ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce035e90-2089-434f-aad9-c172a7244113", "content-length": "163", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117962025-11-12T20:50:36.827ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
117972025-11-12T20:50:36.827ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e52b3c8-aaae-4e48-93a7-abb394dc7157", "content-length": "163", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
117982025-11-12T20:50:36.827ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
117992025-11-12T20:50:36.828ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7d6425b-3f18-49d8-b636-a7646c3a1013", "content-length": "166", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
118002025-11-12T20:50:36.828ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
118012025-11-12T20:50:36.829ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "59a51fb0-abe7-44ed-a43b-e8d618dc081a", "content-length": "166", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
118022025-11-12T20:50:36.829ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
118032025-11-12T20:50:36.829ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8a5979a-c2e0-48af-bf7b-9c2c34910416", "content-length": "702", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
118042025-11-12T20:50:36.829ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
118052025-11-12T20:50:36.829ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03e5d7d7-7244-4968-8ec9-0c2055807325", "content-length": "727", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
118062025-11-12T20:50:36.829ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
118072025-11-12T20:50:36.830ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42b2601f-5f67-41c7-a85e-872136cddbc8", "content-length": "729", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
118082025-11-12T20:50:36.830ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
118092025-11-12T20:50:36.830ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4c22f1d-2772-4eef-9580-96976b39ef79", "content-length": "728", "date": "Wed, 12 Nov 2025 20:50:36 GMT"} })
118102025-11-12T20:50:36.830ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
118112025-11-12T20:50:36.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
118122025-11-12T20:50:36.833ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118132025-11-12T20:50:36.836ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
118142025-11-12T20:50:36.836ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
118152025-11-12T20:50:36.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
118162025-11-12T20:50:36.838ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118172025-11-12T20:50:36.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
118182025-11-12T20:50:36.841ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
118192025-11-12T20:50:36.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
118202025-11-12T20:50:36.959ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
118212025-11-12T20:50:36.959ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
118222025-11-12T20:50:37.017ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4a4f919-b618-4bcc-9059-5652669c132e", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118232025-11-12T20:50:37.019ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
118242025-11-12T20:50:37.019ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
118252025-11-12T20:50:37.019ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
118262025-11-12T20:50:37.019ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
118272025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
118282025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
118292025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
118302025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
118312025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "19632cf4-103e-471a-8611-1999813c53b7", "content-length": "64", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118322025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
118332025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "73f720ef-178d-4ad2-aa0b-76b0e92a25f7", "content-length": "64", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118342025-11-12T20:50:37.020ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
118352025-11-12T20:50:37.021ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "96fe128c-a166-48c3-a7bc-d8a5b7edea26", "content-length": "64", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118362025-11-12T20:50:37.021ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
118372025-11-12T20:50:37.021ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e66f3ee7-c2ee-4b6a-9daa-7bf55fa93aeb", "content-length": "64", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118382025-11-12T20:50:37.021ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
118392025-11-12T20:50:37.021ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a76b992-725b-493f-ba53-63709fd94d0e", "content-length": "64", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118402025-11-12T20:50:37.021ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
118412025-11-12T20:50:37.060ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b73cbb29-36c1-4df3-9566-d44f39980d65", "content-length": "393306", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118422025-11-12T20:50:37.061ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad4e9c9a-6fb4-4f97-ad55-baa69a2b276e", "content-length": "396000", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118432025-11-12T20:50:37.061ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2ef5e66-b8b1-4d76-9083-2bb8380567e3", "content-length": "394797", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118442025-11-12T20:50:37.064ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "65370fb4-b07f-4f5d-802c-0841d99722e1", "content-length": "391476", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118452025-11-12T20:50:37.065ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d36a9e8b-dd52-43d9-8055-8fcf362b0d70", "content-length": "424712", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118462025-11-12T20:50:37.068ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
118472025-11-12T20:50:37.068ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
118482025-11-12T20:50:37.068ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
118492025-11-12T20:50:37.068ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
118502025-11-12T20:50:37.068ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
118512025-11-12T20:50:37.099ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
118522025-11-12T20:50:37.099ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
118532025-11-12T20:50:37.103ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "271b1dd8-49c2-4115-b7b8-cbc3a164291a", "content-length": "127", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118542025-11-12T20:50:37.104ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
118552025-11-12T20:50:37.200ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
118562025-11-12T20:50:37.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
118572025-11-12T20:50:37.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "13a335d9-23a7-4c8e-86ca-8eb0c27be0c9", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118582025-11-12T20:50:37.201ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
118592025-11-12T20:50:37.203ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
118602025-11-12T20:50:37.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
118612025-11-12T20:50:37.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "773a030e-2f6d-4777-92a8-aea1dee2e5a5", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118622025-11-12T20:50:37.203ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
118632025-11-12T20:50:37.204ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
118642025-11-12T20:50:37.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
118652025-11-12T20:50:37.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3868bf0-0be0-43f0-8784-5da02abe7568", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:50:37 GMT"} })
118662025-11-12T20:50:37.205ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
118672025-11-12T20:50:37.206ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
118682025-11-12T20:50:37.206ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
118692025-11-12T20:50:37.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
118702025-11-12T20:50:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
118712025-11-12T20:50:37.210ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118722025-11-12T20:50:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
118732025-11-12T20:50:37.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
118742025-11-12T20:50:37.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
118752025-11-12T20:50:37.417ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 53a0c9f3-40f8-426b-999e-1a06cd803cef file = nexus/db-queries/src/db/datastore/inventory.rs:1614
118762025-11-12T20:50:37.417ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 53a0c9f3-40f8-426b-999e-1a06cd803cef time_started = 2025-11-12 20:50:36.818683 UTC
118772025-11-12T20:50:37.417ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 28 reason = Dependency
118782025-11-12T20:50:37.417ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
118792025-11-12T20:50:37.417ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 700.062665ms iteration = 7 status = {"collection_id":"53a0c9f3-40f8-426b-999e-1a06cd803cef","time_done":"2025-11-12 20:50:37.206519 UTC","time_started":"2025-11-12 20:50:36.818683 UTC"}
118802025-11-12T20:50:37.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
118812025-11-12T20:50:37.420ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118822025-11-12T20:50:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
118832025-11-12T20:50:37.424ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
118842025-11-12T20:50:37.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
118852025-11-12T20:50:37.427ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
118862025-11-12T20:50:37.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
118872025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 49.278695ms iteration = 28 status = {"Loaded":{"collection_id":"a6ce4867-8d96-4d76-9bcd-36258ef2dd87","time_loaded":"2025-11-12T20:50:37.417714055Z","time_started":"2025-11-12T20:50:36.822504Z"}}
118882025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Dependency
118892025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 10 reason = Dependency
118902025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Dependency
118912025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118922025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
118932025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
118942025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118952025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
118962025-11-12T20:50:37.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
118972025-11-12T20:50:37.468ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118982025-11-12T20:50:37.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118992025-11-12T20:50:37.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119002025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
119012025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119022025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
119032025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119042025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119052025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119062025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119072025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119082025-11-12T20:50:37.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119092025-11-12T20:50:37.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119102025-11-12T20:50:37.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119112025-11-12T20:50:37.474ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119122025-11-12T20:50:37.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119132025-11-12T20:50:37.476ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119142025-11-12T20:50:37.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119152025-11-12T20:50:37.476ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119162025-11-12T20:50:37.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119172025-11-12T20:50:37.477ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
119182025-11-12T20:50:37.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119192025-11-12T20:50:37.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119202025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119212025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119222025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119232025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
119242025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119252025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119262025-11-12T20:50:37.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119272025-11-12T20:50:37.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119282025-11-12T20:50:37.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119292025-11-12T20:50:37.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
119302025-11-12T20:50:37.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119312025-11-12T20:50:37.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119322025-11-12T20:50:37.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119332025-11-12T20:50:37.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119342025-11-12T20:50:37.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119352025-11-12T20:50:37.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119362025-11-12T20:50:37.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119372025-11-12T20:50:37.487ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
119382025-11-12T20:50:37.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119392025-11-12T20:50:37.487ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.913391ms iteration = 19 status = {"physical_disks_added":0}
119402025-11-12T20:50:37.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119412025-11-12T20:50:37.488ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119422025-11-12T20:50:37.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119432025-11-12T20:50:37.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119442025-11-12T20:50:37.490ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
119452025-11-12T20:50:37.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119462025-11-12T20:50:37.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119472025-11-12T20:50:37.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119482025-11-12T20:50:37.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
119492025-11-12T20:50:37.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119502025-11-12T20:50:37.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119512025-11-12T20:50:37.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119522025-11-12T20:50:37.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
119532025-11-12T20:50:37.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
119542025-11-12T20:50:37.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
119552025-11-12T20:50:37.503ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
119562025-11-12T20:50:37.503ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 36.211327ms iteration = 10 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"a6ce4867-8d96-4d76-9bcd-36258ef2dd87","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
119572025-11-12T20:50:37.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119582025-11-12T20:50:37.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119592025-11-12T20:50:37.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119602025-11-12T20:50:37.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119612025-11-12T20:50:37.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119622025-11-12T20:50:37.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
119632025-11-12T20:50:37.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119642025-11-12T20:50:37.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119652025-11-12T20:50:37.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119662025-11-12T20:50:37.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119672025-11-12T20:50:37.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119682025-11-12T20:50:37.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
119692025-11-12T20:50:37.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119702025-11-12T20:50:37.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119712025-11-12T20:50:37.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119722025-11-12T20:50:37.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
119732025-11-12T20:50:37.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119742025-11-12T20:50:37.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119752025-11-12T20:50:37.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119762025-11-12T20:50:37.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119772025-11-12T20:50:37.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119782025-11-12T20:50:37.547ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
119792025-11-12T20:50:37.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119802025-11-12T20:50:37.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119812025-11-12T20:50:37.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119822025-11-12T20:50:37.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119832025-11-12T20:50:37.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119842025-11-12T20:50:37.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
119852025-11-12T20:50:37.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119862025-11-12T20:50:37.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119872025-11-12T20:50:37.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119882025-11-12T20:50:37.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119892025-11-12T20:50:37.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119902025-11-12T20:50:37.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
119912025-11-12T20:50:37.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119922025-11-12T20:50:37.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119932025-11-12T20:50:37.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119942025-11-12T20:50:37.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119952025-11-12T20:50:37.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119962025-11-12T20:50:37.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119972025-11-12T20:50:37.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119982025-11-12T20:50:37.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
119992025-11-12T20:50:37.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120002025-11-12T20:50:37.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120012025-11-12T20:50:37.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120022025-11-12T20:50:37.589ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
120032025-11-12T20:50:37.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120042025-11-12T20:50:37.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120052025-11-12T20:50:37.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120062025-11-12T20:50:37.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120072025-11-12T20:50:37.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120082025-11-12T20:50:37.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120092025-11-12T20:50:37.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120102025-11-12T20:50:37.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
120112025-11-12T20:50:37.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120122025-11-12T20:50:37.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120132025-11-12T20:50:37.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120142025-11-12T20:50:37.608ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
120152025-11-12T20:50:37.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120162025-11-12T20:50:37.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120172025-11-12T20:50:37.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120182025-11-12T20:50:37.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120192025-11-12T20:50:37.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120202025-11-12T20:50:37.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120212025-11-12T20:50:37.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120222025-11-12T20:50:37.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
120232025-11-12T20:50:37.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120242025-11-12T20:50:37.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120252025-11-12T20:50:37.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120262025-11-12T20:50:37.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120272025-11-12T20:50:37.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120282025-11-12T20:50:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
120292025-11-12T20:50:37.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120302025-11-12T20:50:37.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120312025-11-12T20:50:37.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120322025-11-12T20:50:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120332025-11-12T20:50:37.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120342025-11-12T20:50:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
120352025-11-12T20:50:37.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120362025-11-12T20:50:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120372025-11-12T20:50:37.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120382025-11-12T20:50:37.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120392025-11-12T20:50:37.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120402025-11-12T20:50:37.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
120412025-11-12T20:50:37.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120422025-11-12T20:50:37.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120432025-11-12T20:50:37.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120442025-11-12T20:50:37.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120452025-11-12T20:50:37.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120462025-11-12T20:50:37.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
120472025-11-12T20:50:37.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120482025-11-12T20:50:37.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120492025-11-12T20:50:37.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120502025-11-12T20:50:37.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120512025-11-12T20:50:37.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120522025-11-12T20:50:37.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
120532025-11-12T20:50:37.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120542025-11-12T20:50:37.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120552025-11-12T20:50:37.658ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
120562025-11-12T20:50:37.658ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
120572025-11-12T20:50:37.658ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 6d7491aa-80f5-4e94-ade1-fe3c806eae3b parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
120582025-11-12T20:50:37.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120592025-11-12T20:50:37.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120602025-11-12T20:50:37.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120612025-11-12T20:50:37.663ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
120622025-11-12T20:50:37.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120632025-11-12T20:50:37.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120642025-11-12T20:50:37.666ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
120652025-11-12T20:50:37.666ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
120662025-11-12T20:50:37.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 199.975576ms iteration = 16 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
120672025-11-12T20:50:37.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 341 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 45eef939-15bb-4963-b5d9-1a6c10f0f2be response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
120682025-11-12T20:50:40.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 62 reason = Timeout
120692025-11-12T20:50:40.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
120702025-11-12T20:50:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
120712025-11-12T20:50:40.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120722025-11-12T20:50:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
120732025-11-12T20:50:40.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
120742025-11-12T20:50:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
120752025-11-12T20:50:40.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
120762025-11-12T20:50:40.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.515045ms iteration = 62 status = {"config_updated":false}
120772025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
120782025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
120792025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120802025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 63 reason = Timeout
120812025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120822025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
120832025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120842025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
120852025-11-12T20:50:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120862025-11-12T20:50:45.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120872025-11-12T20:50:45.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120882025-11-12T20:50:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
120892025-11-12T20:50:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120902025-11-12T20:50:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120912025-11-12T20:50:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
120922025-11-12T20:50:45.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
120932025-11-12T20:50:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
120942025-11-12T20:50:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120952025-11-12T20:50:45.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120962025-11-12T20:50:45.289ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
120972025-11-12T20:50:45.289ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.589628ms iteration = 63 status = {"config_updated":false}
120982025-11-12T20:50:45.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120992025-11-12T20:50:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
121002025-11-12T20:50:45.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121012025-11-12T20:50:45.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
121022025-11-12T20:50:45.311ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
121032025-11-12T20:50:45.311ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.930807ms iteration = 32 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
121042025-11-12T20:50:45.316ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121052025-11-12T20:50:45.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121062025-11-12T20:50:45.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 72.117696ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121072025-11-12T20:50:47.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 424 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 3a7cf12d-c0e7-4397-88aa-83a8022158c0 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
121082025-11-12T20:50:50.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 29 reason = Timeout
121092025-11-12T20:50:50.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
121102025-11-12T20:50:50.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
121112025-11-12T20:50:50.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 64 reason = Timeout
121122025-11-12T20:50:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
121132025-11-12T20:50:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
121142025-11-12T20:50:50.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121152025-11-12T20:50:50.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121162025-11-12T20:50:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
121172025-11-12T20:50:50.284ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
121182025-11-12T20:50:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
121192025-11-12T20:50:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
121202025-11-12T20:50:50.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
121212025-11-12T20:50:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
121222025-11-12T20:50:50.286ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a6ce4867-8d96-4d76-9bcd-36258ef2dd87 original_time_started = 2025-11-12 20:50:36.822504 UTC
121232025-11-12T20:50:50.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.531754ms iteration = 29 status = {"Loaded":{"collection_id":"a6ce4867-8d96-4d76-9bcd-36258ef2dd87","time_loaded":"2025-11-12T20:50:50.278730153Z","time_started":"2025-11-12T20:50:36.822504Z"}}
121242025-11-12T20:50:50.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 22 reason = Timeout
121252025-11-12T20:50:50.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
121262025-11-12T20:50:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
121272025-11-12T20:50:50.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
121282025-11-12T20:50:50.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.670648ms iteration = 64 status = {"config_updated":false}
121292025-11-12T20:50:50.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121302025-11-12T20:50:50.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
121312025-11-12T20:50:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
121322025-11-12T20:50:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
121332025-11-12T20:50:50.292ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.651006ms iteration = 22 status = "NoSitrep"
121342025-11-12T20:50:50.293ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 22 reason = Timeout
121352025-11-12T20:50:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
121362025-11-12T20:50:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
121372025-11-12T20:50:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
121382025-11-12T20:50:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
121392025-11-12T20:50:50.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121402025-11-12T20:50:50.296ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
121412025-11-12T20:50:50.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
121422025-11-12T20:50:50.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
121432025-11-12T20:50:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
121442025-11-12T20:50:50.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
121452025-11-12T20:50:50.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121462025-11-12T20:50:50.302ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
121472025-11-12T20:50:50.303ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
121482025-11-12T20:50:50.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.054561ms iteration = 22 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
121492025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
121502025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
121512025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121522025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121532025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 65 reason = Timeout
121542025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
121552025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121562025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
121572025-11-12T20:50:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
121582025-11-12T20:50:55.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121592025-11-12T20:50:55.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121602025-11-12T20:50:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
121612025-11-12T20:50:55.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
121622025-11-12T20:50:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
121632025-11-12T20:50:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
121642025-11-12T20:50:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121652025-11-12T20:50:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
121662025-11-12T20:50:55.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
121672025-11-12T20:50:55.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.108994ms iteration = 65 status = {"config_updated":false}
121682025-11-12T20:50:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121692025-11-12T20:50:55.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
121702025-11-12T20:50:55.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121712025-11-12T20:50:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
121722025-11-12T20:50:55.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121732025-11-12T20:50:55.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
121742025-11-12T20:50:55.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
121752025-11-12T20:50:55.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.614527ms iteration = 33 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
121762025-11-12T20:50:55.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121772025-11-12T20:50:55.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121782025-11-12T20:50:55.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.384212ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121792025-11-12T20:50:57.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 269 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 9f27ffba-3fe1-44c2-b3dc-c72013bbfcb7 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
121802025-11-12T20:51:00.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 66 reason = Timeout
121812025-11-12T20:51:00.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
121822025-11-12T20:51:00.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
121832025-11-12T20:51:00.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121842025-11-12T20:51:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
121852025-11-12T20:51:00.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
121862025-11-12T20:51:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
121872025-11-12T20:51:00.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
121882025-11-12T20:51:00.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.150345ms iteration = 66 status = {"config_updated":false}
121892025-11-12T20:51:05.276ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
121902025-11-12T20:51:05.276ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 12 reason = Timeout
121912025-11-12T20:51:05.276ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
121922025-11-12T20:51:05.276ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
121932025-11-12T20:51:05.276ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
121942025-11-12T20:51:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
121952025-11-12T20:51:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
121962025-11-12T20:51:05.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 30 reason = Timeout
121972025-11-12T20:51:05.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
121982025-11-12T20:51:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
121992025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
122002025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
122012025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
122022025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
122032025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
122042025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
122052025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
122062025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
122072025-11-12T20:51:05.279ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
122082025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 12 reason = Timeout
122092025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
122102025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122112025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122122025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
122132025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
122142025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
122152025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
122162025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
122172025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
122182025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122192025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
122202025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
122212025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
122222025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
122232025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122242025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
122252025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
122262025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122272025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122282025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 67 reason = Timeout
122292025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
122302025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
122312025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122322025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
122332025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
122342025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
122352025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
122362025-11-12T20:51:05.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1260, tv_nsec: 774790648 } })
122372025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
122382025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
122392025-11-12T20:51:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
122402025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
122412025-11-12T20:51:05.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1260, tv_nsec: 775000573 } })
122422025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
122432025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
122442025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
122452025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122462025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122472025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
122482025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122492025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
122502025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122512025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122522025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122532025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
122542025-11-12T20:51:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
122552025-11-12T20:51:05.280ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1260, tv_nsec: 775835845 } })
122562025-11-12T20:51:05.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
122572025-11-12T20:51:05.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122582025-11-12T20:51:05.282ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
122592025-11-12T20:51:05.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
122602025-11-12T20:51:05.283ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122612025-11-12T20:51:05.284ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122622025-11-12T20:51:05.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122632025-11-12T20:51:05.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 23 reason = Timeout
122642025-11-12T20:51:05.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
122652025-11-12T20:51:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
122662025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
122672025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
122682025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
122692025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122702025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
122712025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
122722025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
122732025-11-12T20:51:05.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
122742025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122752025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122762025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
122772025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
122782025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
122792025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122802025-11-12T20:51:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122812025-11-12T20:51:05.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 23 reason = Timeout
122822025-11-12T20:51:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
122832025-11-12T20:51:05.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
122842025-11-12T20:51:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
122852025-11-12T20:51:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
122862025-11-12T20:51:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
122872025-11-12T20:51:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
122882025-11-12T20:51:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.709148ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
122892025-11-12T20:51:05.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122902025-11-12T20:51:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
122912025-11-12T20:51:05.297ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a6ce4867-8d96-4d76-9bcd-36258ef2dd87 original_time_started = 2025-11-12 20:50:36.822504 UTC
122922025-11-12T20:51:05.297ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 19.111925ms iteration = 30 status = {"Loaded":{"collection_id":"a6ce4867-8d96-4d76-9bcd-36258ef2dd87","time_loaded":"2025-11-12T20:51:05.278116837Z","time_started":"2025-11-12T20:50:36.822504Z"}}
122932025-11-12T20:51:05.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
122942025-11-12T20:51:05.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122952025-11-12T20:51:05.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122962025-11-12T20:51:05.307ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 20.844639ms iteration = 23 status = "NoSitrep"
122972025-11-12T20:51:05.314ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122982025-11-12T20:51:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
122992025-11-12T20:51:05.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
123002025-11-12T20:51:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
123012025-11-12T20:51:05.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
123022025-11-12T20:51:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
123032025-11-12T20:51:05.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123042025-11-12T20:51:05.328ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
123052025-11-12T20:51:05.329ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
123062025-11-12T20:51:05.329ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 36.754361ms iteration = 23 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
123072025-11-12T20:51:05.358ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123082025-11-12T20:51:05.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123092025-11-12T20:51:05.360ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
123102025-11-12T20:51:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
123112025-11-12T20:51:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
123122025-11-12T20:51:05.362ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
123132025-11-12T20:51:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
123142025-11-12T20:51:05.362ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.567202ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
123152025-11-12T20:51:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
123162025-11-12T20:51:05.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123172025-11-12T20:51:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123182025-11-12T20:51:05.367ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123192025-11-12T20:51:05.367ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123202025-11-12T20:51:05.367ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
123212025-11-12T20:51:05.367ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
123222025-11-12T20:51:05.367ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123232025-11-12T20:51:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
123242025-11-12T20:51:05.368ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1260, tv_nsec: 863516164 } })
123252025-11-12T20:51:05.369ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
123262025-11-12T20:51:05.369ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 90.379169ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
123272025-11-12T20:51:05.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123282025-11-12T20:51:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123292025-11-12T20:51:05.371ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123302025-11-12T20:51:05.371ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
123312025-11-12T20:51:05.371ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
123322025-11-12T20:51:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
123332025-11-12T20:51:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
123342025-11-12T20:51:05.373ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
123352025-11-12T20:51:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
123362025-11-12T20:51:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
123372025-11-12T20:51:05.373ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
123382025-11-12T20:51:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
123392025-11-12T20:51:05.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123402025-11-12T20:51:05.378ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
123412025-11-12T20:51:05.378ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 98.960271ms iteration = 67 status = {"config_updated":false}
123422025-11-12T20:51:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
123432025-11-12T20:51:05.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123442025-11-12T20:51:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123452025-11-12T20:51:05.380ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123462025-11-12T20:51:05.381ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
123472025-11-12T20:51:05.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
123482025-11-12T20:51:05.381ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
123492025-11-12T20:51:05.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
123502025-11-12T20:51:05.382ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 102.710788ms iteration = 12 status = 0
123512025-11-12T20:51:05.383ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
123522025-11-12T20:51:05.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
123532025-11-12T20:51:05.383ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
123542025-11-12T20:51:05.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
123552025-11-12T20:51:05.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
123562025-11-12T20:51:05.384ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 105.298915ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
123572025-11-12T20:51:05.385ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123582025-11-12T20:51:05.386ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
123592025-11-12T20:51:05.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
123602025-11-12T20:51:05.387ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
123612025-11-12T20:51:05.387ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 108.110598ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
123622025-11-12T20:51:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
123632025-11-12T20:51:05.388ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
123642025-11-12T20:51:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
123652025-11-12T20:51:05.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 114.056092ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
123662025-11-12T20:51:05.402ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
123672025-11-12T20:51:05.402ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 123.182177ms iteration = 34 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
123682025-11-12T20:51:05.411ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
123692025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c8602f8-28a3-4eb3-b719-1075af280a4f", "content-length": "78", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
123702025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
123712025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
123722025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
123732025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
123742025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
123752025-11-12T20:51:05.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
123762025-11-12T20:51:05.413ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
123772025-11-12T20:51:05.413ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
123782025-11-12T20:51:05.413ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2f2bbeb-8ff0-4155-99e9-f85d12d0188c", "content-length": "26", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
123792025-11-12T20:51:05.413ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
123802025-11-12T20:51:05.414ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
123812025-11-12T20:51:05.414ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
123822025-11-12T20:51:05.415ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
123832025-11-12T20:51:05.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
123842025-11-12T20:51:05.418ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
123852025-11-12T20:51:05.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
123862025-11-12T20:51:05.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
123872025-11-12T20:51:05.419ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
123882025-11-12T20:51:05.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
123892025-11-12T20:51:05.439ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 160.414541ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
123902025-11-12T20:51:05.441ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
123912025-11-12T20:51:05.441ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
123922025-11-12T20:51:05.441ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 161.673454ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
123932025-11-12T20:51:05.445ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123942025-11-12T20:51:05.445ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
123952025-11-12T20:51:05.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
123962025-11-12T20:51:05.447ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 167.824493ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
123972025-11-12T20:51:05.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
123982025-11-12T20:51:05.449ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
123992025-11-12T20:51:05.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
124002025-11-12T20:51:05.455ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
124012025-11-12T20:51:05.455ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 175.680747ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
124022025-11-12T20:51:05.458ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
124032025-11-12T20:51:05.460ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 183.92635ms iteration = 12 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
124042025-11-12T20:51:05.468ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
124052025-11-12T20:51:05.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
124062025-11-12T20:51:05.472ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
124072025-11-12T20:51:05.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
124082025-11-12T20:51:05.474ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
124092025-11-12T20:51:05.474ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
124102025-11-12T20:51:05.474ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "39ba852a-4009-45c7-98f4-1c9b3ff6af16", "content-length": "81", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124112025-11-12T20:51:05.478ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
124122025-11-12T20:51:05.478ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
124132025-11-12T20:51:05.478ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "85fb16a8-9196-4cc4-98b6-fcb60b1e4cc6", "content-length": "81", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124142025-11-12T20:51:05.480ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
124152025-11-12T20:51:05.481ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
124162025-11-12T20:51:05.481ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03f0054b-7250-4b04-88e6-6c0cd1fbac1c", "content-length": "26", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124172025-11-12T20:51:05.481ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
124182025-11-12T20:51:05.482ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
124192025-11-12T20:51:05.493ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124202025-11-12T20:51:05.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
124212025-11-12T20:51:05.493ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 214.180373ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
124222025-11-12T20:51:05.516ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 240.013251ms iteration = 12 status = {}
124232025-11-12T20:51:05.528ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
124242025-11-12T20:51:05.528ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
124252025-11-12T20:51:05.528ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9b24ad3-6a3f-4780-b344-e10935bb46bf", "content-length": "81", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124262025-11-12T20:51:05.531ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 251.707704ms iteration = 12 status = {"disabled":false,"errors":[],"sps":[]}
124272025-11-12T20:51:05.537ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
124282025-11-12T20:51:05.537ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124292025-11-12T20:51:05.542ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124302025-11-12T20:51:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
124312025-11-12T20:51:05.543ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e366eda9-bfbf-45a2-8d2b-9474c4a0702a", "content-length": "372", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124322025-11-12T20:51:05.543ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124332025-11-12T20:51:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124342025-11-12T20:51:05.548ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 43039030 } })
124352025-11-12T20:51:05.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124362025-11-12T20:51:05.551ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
124372025-11-12T20:51:05.551ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124382025-11-12T20:51:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
124392025-11-12T20:51:05.553ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124402025-11-12T20:51:05.555ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
124412025-11-12T20:51:05.555ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124422025-11-12T20:51:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
124432025-11-12T20:51:05.596ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 317.132677ms iteration = 20 status = {"physical_disks_added":0}
124442025-11-12T20:51:05.597ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
124452025-11-12T20:51:05.597ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
124462025-11-12T20:51:05.598ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3abf6053-1627-440e-8b9b-db3dca81da59", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124472025-11-12T20:51:05.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124482025-11-12T20:51:05.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124492025-11-12T20:51:05.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124502025-11-12T20:51:05.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124512025-11-12T20:51:05.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124522025-11-12T20:51:05.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124532025-11-12T20:51:05.608ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
124542025-11-12T20:51:05.608ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124552025-11-12T20:51:05.609ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ffefee8c-a83e-4465-80cd-3046a89edbb3", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124562025-11-12T20:51:05.611ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 331.701825ms iteration = 18 status = {}
124572025-11-12T20:51:05.614ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124582025-11-12T20:51:05.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124592025-11-12T20:51:05.617ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
124602025-11-12T20:51:05.617ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124612025-11-12T20:51:05.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124622025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
124632025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
124642025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
124652025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
124662025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124672025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124682025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124692025-11-12T20:51:05.630ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
124702025-11-12T20:51:05.631ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
124712025-11-12T20:51:05.631ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
124722025-11-12T20:51:05.631ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
124732025-11-12T20:51:05.631ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124742025-11-12T20:51:05.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124752025-11-12T20:51:05.633ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
124762025-11-12T20:51:05.633ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
124772025-11-12T20:51:05.661ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
124782025-11-12T20:51:05.665ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "10435a08-9747-4a03-bd6f-a84fb11de120", "content-length": "691", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124792025-11-12T20:51:05.665ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
124802025-11-12T20:51:05.666ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
124812025-11-12T20:51:05.666ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
124822025-11-12T20:51:05.667ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 388.147255ms iteration = 12 status = {}
124832025-11-12T20:51:05.814ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "965714fa-afb8-438a-914d-6973b1a0ab9d", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124842025-11-12T20:51:05.814ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
124852025-11-12T20:51:05.814ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
124862025-11-12T20:51:05.815ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "146633c8-1266-4aa0-82de-2748dbe66991", "content-length": "682", "date": "Wed, 12 Nov 2025 20:51:05 GMT"} })
124872025-11-12T20:51:05.815ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124882025-11-12T20:51:05.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124892025-11-12T20:51:05.819ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124902025-11-12T20:51:05.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124912025-11-12T20:51:05.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124922025-11-12T20:51:05.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124932025-11-12T20:51:05.824ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124942025-11-12T20:51:05.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124952025-11-12T20:51:05.832ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 552.56613ms iteration = 12 status = {}
124962025-11-12T20:51:07.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 418 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 197cfc04-542a-431d-bb6a-c629a0a0ef2a response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
124972025-11-12T20:51:10.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 68 reason = Timeout
124982025-11-12T20:51:10.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124992025-11-12T20:51:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125002025-11-12T20:51:10.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125012025-11-12T20:51:10.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
125022025-11-12T20:51:10.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125032025-11-12T20:51:10.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125042025-11-12T20:51:10.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
125052025-11-12T20:51:10.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.814226ms iteration = 68 status = {"config_updated":false}
125062025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
125072025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125082025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
125092025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 69 reason = Timeout
125102025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125112025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125122025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125132025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125142025-11-12T20:51:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125152025-11-12T20:51:15.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125162025-11-12T20:51:15.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125172025-11-12T20:51:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
125182025-11-12T20:51:15.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125192025-11-12T20:51:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
125202025-11-12T20:51:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125212025-11-12T20:51:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125222025-11-12T20:51:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125232025-11-12T20:51:15.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125242025-11-12T20:51:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125252025-11-12T20:51:15.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
125262025-11-12T20:51:15.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.48159ms iteration = 69 status = {"config_updated":false}
125272025-11-12T20:51:15.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125282025-11-12T20:51:15.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125292025-11-12T20:51:15.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.02162ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
125302025-11-12T20:51:15.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125312025-11-12T20:51:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
125322025-11-12T20:51:15.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125332025-11-12T20:51:15.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125342025-11-12T20:51:15.342ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
125352025-11-12T20:51:15.342ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 62.147618ms iteration = 35 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
125362025-11-12T20:51:17.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 19693ad1-0be2-4477-b701-53d190bb7fc8 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
125372025-11-12T20:51:20.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 31 reason = Timeout
125382025-11-12T20:51:20.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
125392025-11-12T20:51:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
125402025-11-12T20:51:20.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 70 reason = Timeout
125412025-11-12T20:51:20.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125422025-11-12T20:51:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125432025-11-12T20:51:20.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125442025-11-12T20:51:20.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125452025-11-12T20:51:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
125462025-11-12T20:51:20.284ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
125472025-11-12T20:51:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
125482025-11-12T20:51:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
125492025-11-12T20:51:20.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125502025-11-12T20:51:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125512025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a6ce4867-8d96-4d76-9bcd-36258ef2dd87 original_time_started = 2025-11-12 20:50:36.822504 UTC
125522025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.757891ms iteration = 31 status = {"Loaded":{"collection_id":"a6ce4867-8d96-4d76-9bcd-36258ef2dd87","time_loaded":"2025-11-12T20:51:20.278563686Z","time_started":"2025-11-12T20:50:36.822504Z"}}
125532025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 24 reason = Timeout
125542025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
125552025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
125562025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
125572025-11-12T20:51:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.751311ms iteration = 70 status = {"config_updated":false}
125582025-11-12T20:51:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125592025-11-12T20:51:20.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
125602025-11-12T20:51:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
125612025-11-12T20:51:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
125622025-11-12T20:51:20.291ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.221696ms iteration = 24 status = "NoSitrep"
125632025-11-12T20:51:20.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 24 reason = Timeout
125642025-11-12T20:51:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
125652025-11-12T20:51:20.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
125662025-11-12T20:51:20.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
125672025-11-12T20:51:20.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
125682025-11-12T20:51:20.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125692025-11-12T20:51:20.296ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
125702025-11-12T20:51:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
125712025-11-12T20:51:20.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
125722025-11-12T20:51:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
125732025-11-12T20:51:20.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
125742025-11-12T20:51:20.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125752025-11-12T20:51:20.302ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
125762025-11-12T20:51:20.303ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
125772025-11-12T20:51:20.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.284656ms iteration = 24 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
125782025-11-12T20:51:25.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
125792025-11-12T20:51:25.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
125802025-11-12T20:51:25.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
125812025-11-12T20:51:25.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125822025-11-12T20:51:25.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 71 reason = Timeout
125832025-11-12T20:51:25.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125842025-11-12T20:51:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
125852025-11-12T20:51:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125862025-11-12T20:51:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125872025-11-12T20:51:25.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125882025-11-12T20:51:25.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125892025-11-12T20:51:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
125902025-11-12T20:51:25.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125912025-11-12T20:51:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
125922025-11-12T20:51:25.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125932025-11-12T20:51:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125942025-11-12T20:51:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125952025-11-12T20:51:25.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125962025-11-12T20:51:25.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
125972025-11-12T20:51:25.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.956876ms iteration = 71 status = {"config_updated":false}
125982025-11-12T20:51:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125992025-11-12T20:51:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126002025-11-12T20:51:25.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
126012025-11-12T20:51:25.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
126022025-11-12T20:51:25.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
126032025-11-12T20:51:25.309ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
126042025-11-12T20:51:25.309ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.37335ms iteration = 36 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
126052025-11-12T20:51:25.312ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
126062025-11-12T20:51:25.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
126072025-11-12T20:51:25.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.926084ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
126082025-11-12T20:51:27.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 298 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 1d35e689-23f1-4449-a25e-c3bfd2a126a5 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
126092025-11-12T20:51:30.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 72 reason = Timeout
126102025-11-12T20:51:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
126112025-11-12T20:51:30.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
126122025-11-12T20:51:30.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126132025-11-12T20:51:30.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
126142025-11-12T20:51:30.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
126152025-11-12T20:51:30.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
126162025-11-12T20:51:30.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
126172025-11-12T20:51:30.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.866128ms iteration = 72 status = {"config_updated":false}
126182025-11-12T20:51:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
126192025-11-12T20:51:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
126202025-11-12T20:51:35.276ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
126212025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
126222025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
126232025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
126242025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126252025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
126262025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
126272025-11-12T20:51:35.277ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
126282025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
126292025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126302025-11-12T20:51:35.277ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = ExternalNetworking step id = Ensure
126312025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
126322025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
126332025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126342025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126352025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 13 reason = Timeout
126362025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
126372025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
126382025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
126392025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
126402025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
126412025-11-12T20:51:35.277ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:41:35.277121600 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
126422025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
126432025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
126442025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126452025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
126462025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126472025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
126482025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
126492025-11-12T20:51:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
126502025-11-12T20:51:35.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1290, tv_nsec: 772795952 } })
126512025-11-12T20:51:35.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1290, tv_nsec: 773008168 } })
126522025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 32 reason = Timeout
126532025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
126542025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
126552025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.235152ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
126562025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
126572025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.712165ms iteration = 7 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
126582025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
126592025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126602025-11-12T20:51:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126612025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
126622025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
126632025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
126642025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
126652025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
126662025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
126672025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
126682025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126692025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126702025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
126712025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
126722025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126732025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
126742025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
126752025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
126762025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
126772025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126782025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
126792025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 13 reason = Timeout
126802025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
126812025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
126822025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
126832025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
126842025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
126852025-11-12T20:51:35.279ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
126862025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
126872025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
126882025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
126892025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 73 reason = Timeout
126902025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
126912025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
126922025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
126932025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
126942025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
126952025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
126962025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
126972025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
126982025-11-12T20:51:35.279ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
126992025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
127002025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
127012025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
127022025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
127032025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
127042025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
127052025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
127062025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
127072025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
127082025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
127092025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
127102025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
127112025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
127122025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
127132025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Timeout
127142025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
127152025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
127162025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
127172025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127182025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
127192025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127202025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
127212025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
127222025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127232025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
127242025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
127252025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
127262025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127272025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127282025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
127292025-11-12T20:51:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1290, tv_nsec: 774932347 } })
127302025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
127312025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
127322025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
127332025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
127342025-11-12T20:51:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
127352025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
127362025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
127372025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
127382025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
127392025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
127402025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
127412025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
127422025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127432025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
127442025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
127452025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127462025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127472025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127482025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127492025-11-12T20:51:35.280ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127502025-11-12T20:51:35.281ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127512025-11-12T20:51:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127522025-11-12T20:51:35.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127532025-11-12T20:51:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127542025-11-12T20:51:35.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127552025-11-12T20:51:35.282ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1290, tv_nsec: 777233017 } })
127562025-11-12T20:51:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127572025-11-12T20:51:35.282ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
127582025-11-12T20:51:35.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127592025-11-12T20:51:35.283ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127602025-11-12T20:51:35.284ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
127612025-11-12T20:51:35.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127622025-11-12T20:51:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
127632025-11-12T20:51:35.285ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:41:35.277121600 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
127642025-11-12T20:51:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.575912ms iteration = 7 status = {"expiration":"2025-11-12T20:41:35.277121600Z","pruned":[]}
127652025-11-12T20:51:35.285ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
127662025-11-12T20:51:35.285ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
127672025-11-12T20:51:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
127682025-11-12T20:51:35.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 25 reason = Timeout
127692025-11-12T20:51:35.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
127702025-11-12T20:51:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
127712025-11-12T20:51:35.287ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1290, tv_nsec: 782624936 } })
127722025-11-12T20:51:35.287ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.677086ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
127732025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
127742025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
127752025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
127762025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127772025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
127782025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
127792025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
127802025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
127812025-11-12T20:51:35.288ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
127822025-11-12T20:51:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
127832025-11-12T20:51:35.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127842025-11-12T20:51:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
127852025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 7 reason = Timeout
127862025-11-12T20:51:35.292ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
127872025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 25 reason = Timeout
127882025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
127892025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
127902025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
127912025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
127922025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
127932025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
127942025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
127952025-11-12T20:51:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
127962025-11-12T20:51:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
127972025-11-12T20:51:35.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
127982025-11-12T20:51:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
127992025-11-12T20:51:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
128002025-11-12T20:51:35.293ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
128012025-11-12T20:51:35.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128022025-11-12T20:51:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128032025-11-12T20:51:35.297ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
128042025-11-12T20:51:35.297ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 18.183041ms iteration = 73 status = {"config_updated":false}
128052025-11-12T20:51:35.298ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128062025-11-12T20:51:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128072025-11-12T20:51:35.301ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
128082025-11-12T20:51:35.301ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
128092025-11-12T20:51:35.301ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 21.807074ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
128102025-11-12T20:51:35.306ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128112025-11-12T20:51:35.307ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 28.384714ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
128122025-11-12T20:51:35.309ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128132025-11-12T20:51:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
128142025-11-12T20:51:35.310ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
128152025-11-12T20:51:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
128162025-11-12T20:51:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
128172025-11-12T20:51:35.313ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
128182025-11-12T20:51:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
128192025-11-12T20:51:35.325ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
128202025-11-12T20:51:35.325ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 46.135444ms iteration = 7 status = {"by_rx":{},"error":null}
128212025-11-12T20:51:35.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128222025-11-12T20:51:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128232025-11-12T20:51:35.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
128242025-11-12T20:51:35.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128252025-11-12T20:51:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128262025-11-12T20:51:35.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
128272025-11-12T20:51:35.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
128282025-11-12T20:51:35.333ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128292025-11-12T20:51:35.334ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
128302025-11-12T20:51:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
128312025-11-12T20:51:35.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128322025-11-12T20:51:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128332025-11-12T20:51:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
128342025-11-12T20:51:35.335ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
128352025-11-12T20:51:35.336ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128362025-11-12T20:51:35.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
128372025-11-12T20:51:35.338ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a6ce4867-8d96-4d76-9bcd-36258ef2dd87 original_time_started = 2025-11-12 20:50:36.822504 UTC
128382025-11-12T20:51:35.338ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 60.405163ms iteration = 32 status = {"Loaded":{"collection_id":"a6ce4867-8d96-4d76-9bcd-36258ef2dd87","time_loaded":"2025-11-12T20:51:35.278122596Z","time_started":"2025-11-12T20:50:36.822504Z"}}
128392025-11-12T20:51:35.338ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128402025-11-12T20:51:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128412025-11-12T20:51:35.339ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
128422025-11-12T20:51:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
128432025-11-12T20:51:35.339ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
128442025-11-12T20:51:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
128452025-11-12T20:51:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
128462025-11-12T20:51:35.340ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
128472025-11-12T20:51:35.340ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
128482025-11-12T20:51:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
128492025-11-12T20:51:35.341ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128502025-11-12T20:51:35.345ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
128512025-11-12T20:51:35.345ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 66.46088ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
128522025-11-12T20:51:35.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
128532025-11-12T20:51:35.347ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
128542025-11-12T20:51:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
128552025-11-12T20:51:35.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128562025-11-12T20:51:35.352ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 73.611675ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
128572025-11-12T20:51:35.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
128582025-11-12T20:51:35.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128592025-11-12T20:51:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128602025-11-12T20:51:35.363ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
128612025-11-12T20:51:35.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
128622025-11-12T20:51:35.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128632025-11-12T20:51:35.368ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128642025-11-12T20:51:35.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
128652025-11-12T20:51:35.372ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
128662025-11-12T20:51:35.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
128672025-11-12T20:51:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
128682025-11-12T20:51:35.380ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128692025-11-12T20:51:35.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128702025-11-12T20:51:35.383ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128712025-11-12T20:51:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
128722025-11-12T20:51:35.385ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
128732025-11-12T20:51:35.385ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128742025-11-12T20:51:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
128752025-11-12T20:51:35.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128762025-11-12T20:51:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
128772025-11-12T20:51:35.387ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128782025-11-12T20:51:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128792025-11-12T20:51:35.387ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128802025-11-12T20:51:35.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
128812025-11-12T20:51:35.388ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
128822025-11-12T20:51:35.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
128832025-11-12T20:51:35.390ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
128842025-11-12T20:51:35.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
128852025-11-12T20:51:35.391ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
128862025-11-12T20:51:35.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
128872025-11-12T20:51:35.391ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128882025-11-12T20:51:35.392ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
128892025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
128902025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
128912025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128922025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
128932025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
128942025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
128952025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 113.395844ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
128962025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128972025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
128982025-11-12T20:51:35.392ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 105.981382ms iteration = 25 status = "NoSitrep"
128992025-11-12T20:51:35.393ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
129002025-11-12T20:51:35.394ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
129012025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
129022025-11-12T20:51:35.395ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
129032025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
129042025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
129052025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129062025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
129072025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129082025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.983561ms iteration = 13 status = 0
129092025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
129102025-11-12T20:51:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
129112025-11-12T20:51:35.397ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
129122025-11-12T20:51:35.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129132025-11-12T20:51:35.397ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
129142025-11-12T20:51:35.397ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
129152025-11-12T20:51:35.397ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
129162025-11-12T20:51:35.397ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
129172025-11-12T20:51:35.397ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
129182025-11-12T20:51:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
129192025-11-12T20:51:35.398ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
129202025-11-12T20:51:35.398ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
129212025-11-12T20:51:35.398ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
129222025-11-12T20:51:35.398ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
129232025-11-12T20:51:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
129242025-11-12T20:51:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
129252025-11-12T20:51:35.399ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1290, tv_nsec: 894792869 } })
129262025-11-12T20:51:35.399ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
129272025-11-12T20:51:35.399ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
129282025-11-12T20:51:35.399ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
129292025-11-12T20:51:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
129302025-11-12T20:51:35.400ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
129312025-11-12T20:51:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
129322025-11-12T20:51:35.400ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
129332025-11-12T20:51:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
129342025-11-12T20:51:35.401ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129352025-11-12T20:51:35.401ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129362025-11-12T20:51:35.401ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
129372025-11-12T20:51:35.401ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
129382025-11-12T20:51:35.402ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
129392025-11-12T20:51:35.402ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 123.078935ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
129402025-11-12T20:51:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
129412025-11-12T20:51:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
129422025-11-12T20:51:35.404ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
129432025-11-12T20:51:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
129442025-11-12T20:51:35.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
129452025-11-12T20:51:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
129462025-11-12T20:51:35.406ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
129472025-11-12T20:51:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
129482025-11-12T20:51:35.406ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
129492025-11-12T20:51:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
129502025-11-12T20:51:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
129512025-11-12T20:51:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
129522025-11-12T20:51:35.408ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
129532025-11-12T20:51:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
129542025-11-12T20:51:35.409ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129552025-11-12T20:51:35.410ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129562025-11-12T20:51:35.410ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
129572025-11-12T20:51:35.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 131.522474ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
129582025-11-12T20:51:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 132.679673ms iteration = 4 status = {"nsuccess":5}
129592025-11-12T20:51:35.412ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
129602025-11-12T20:51:35.412ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129612025-11-12T20:51:35.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
129622025-11-12T20:51:35.414ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
129632025-11-12T20:51:35.414ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
129642025-11-12T20:51:35.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
129652025-11-12T20:51:35.415ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
129662025-11-12T20:51:35.415ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 123.196078ms iteration = 25 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
129672025-11-12T20:51:35.416ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
129682025-11-12T20:51:35.416ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = SledList step id = Fetch
129692025-11-12T20:51:35.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
129702025-11-12T20:51:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
129712025-11-12T20:51:35.417ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
129722025-11-12T20:51:35.417ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
129732025-11-12T20:51:35.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
129742025-11-12T20:51:35.417ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 138.630348ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
129752025-11-12T20:51:35.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129762025-11-12T20:51:35.426ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
129772025-11-12T20:51:35.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
129782025-11-12T20:51:35.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
129792025-11-12T20:51:35.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129802025-11-12T20:51:35.432ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
129812025-11-12T20:51:35.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
129822025-11-12T20:51:35.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
129832025-11-12T20:51:35.459ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
129842025-11-12T20:51:35.460ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
129852025-11-12T20:51:35.460ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 180.680906ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
129862025-11-12T20:51:35.460ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
129872025-11-12T20:51:35.460ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 181.332893ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
129882025-11-12T20:51:35.473ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
129892025-11-12T20:51:35.473ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 194.529015ms iteration = 37 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
129902025-11-12T20:51:35.480ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
129912025-11-12T20:51:35.480ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
129922025-11-12T20:51:35.480ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
129932025-11-12T20:51:35.480ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
129942025-11-12T20:51:35.480ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 201.811053ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
129952025-11-12T20:51:35.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
129962025-11-12T20:51:35.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129972025-11-12T20:51:35.488ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
129982025-11-12T20:51:35.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
129992025-11-12T20:51:35.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
130002025-11-12T20:51:35.490ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
130012025-11-12T20:51:35.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
130022025-11-12T20:51:35.493ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
130032025-11-12T20:51:35.493ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
130042025-11-12T20:51:35.493ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 214.8215ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
130052025-11-12T20:51:35.496ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
130062025-11-12T20:51:35.496ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
130072025-11-12T20:51:35.496ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
130082025-11-12T20:51:35.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
130092025-11-12T20:51:35.500ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 221.081091ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
130102025-11-12T20:51:35.502ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
130112025-11-12T20:51:35.502ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 225.407423ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
130122025-11-12T20:51:35.504ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130132025-11-12T20:51:35.507ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
130142025-11-12T20:51:35.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130152025-11-12T20:51:35.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
130162025-11-12T20:51:35.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
130172025-11-12T20:51:35.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
130182025-11-12T20:51:35.511ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130192025-11-12T20:51:35.513ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
130202025-11-12T20:51:35.513ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
130212025-11-12T20:51:35.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
130222025-11-12T20:51:35.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
130232025-11-12T20:51:35.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
130242025-11-12T20:51:35.515ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
130252025-11-12T20:51:35.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130262025-11-12T20:51:35.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130272025-11-12T20:51:35.516ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
130282025-11-12T20:51:35.516ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
130292025-11-12T20:51:35.516ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
130302025-11-12T20:51:35.516ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 239.617291ms iteration = 7 status = {"generation":2}
130312025-11-12T20:51:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
130322025-11-12T20:51:35.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
130332025-11-12T20:51:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
130342025-11-12T20:51:35.529ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
130352025-11-12T20:51:35.529ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
130362025-11-12T20:51:35.529ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
130372025-11-12T20:51:35.529ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 252.473884ms iteration = 7 status = {"generation":1}
130382025-11-12T20:51:35.551ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
130392025-11-12T20:51:35.551ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3012cfc-2564-475a-a004-32734b0c7eb1", "content-length": "78", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
130402025-11-12T20:51:35.551ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
130412025-11-12T20:51:35.551ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
130422025-11-12T20:51:35.551ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
130432025-11-12T20:51:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
130442025-11-12T20:51:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
130452025-11-12T20:51:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
130462025-11-12T20:51:35.554ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
130472025-11-12T20:51:35.558ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
130482025-11-12T20:51:35.558ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
130492025-11-12T20:51:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
130502025-11-12T20:51:35.559ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
130512025-11-12T20:51:35.560ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
130522025-11-12T20:51:35.562ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
130532025-11-12T20:51:35.563ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
130542025-11-12T20:51:35.563ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
130552025-11-12T20:51:35.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
130562025-11-12T20:51:35.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
130572025-11-12T20:51:35.564ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
130582025-11-12T20:51:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
130592025-11-12T20:51:35.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130602025-11-12T20:51:35.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130612025-11-12T20:51:35.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130622025-11-12T20:51:35.576ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 297.781656ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
130632025-11-12T20:51:35.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
130642025-11-12T20:51:35.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130652025-11-12T20:51:35.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130662025-11-12T20:51:35.579ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
130672025-11-12T20:51:35.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
130682025-11-12T20:51:35.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130692025-11-12T20:51:35.585ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
130702025-11-12T20:51:35.585ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 306.215044ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
130712025-11-12T20:51:35.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
130722025-11-12T20:51:35.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130732025-11-12T20:51:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130742025-11-12T20:51:35.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130752025-11-12T20:51:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130762025-11-12T20:51:35.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130772025-11-12T20:51:35.603ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = DeployNexusRecords step id = Ensure
130782025-11-12T20:51:35.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
130792025-11-12T20:51:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130802025-11-12T20:51:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
130812025-11-12T20:51:35.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130822025-11-12T20:51:35.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130832025-11-12T20:51:35.608ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
130842025-11-12T20:51:35.608ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
130852025-11-12T20:51:35.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
130862025-11-12T20:51:35.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
130872025-11-12T20:51:35.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
130882025-11-12T20:51:35.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
130892025-11-12T20:51:35.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
130902025-11-12T20:51:35.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130912025-11-12T20:51:35.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130922025-11-12T20:51:35.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130932025-11-12T20:51:35.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130942025-11-12T20:51:35.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130952025-11-12T20:51:35.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
130962025-11-12T20:51:35.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130972025-11-12T20:51:35.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
130982025-11-12T20:51:35.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130992025-11-12T20:51:35.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131002025-11-12T20:51:35.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131012025-11-12T20:51:35.633ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
131022025-11-12T20:51:35.633ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
131032025-11-12T20:51:35.633ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ae82a6b-2734-4a6c-8084-04d9e2041085", "content-length": "81", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131042025-11-12T20:51:35.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
131052025-11-12T20:51:35.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131062025-11-12T20:51:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131072025-11-12T20:51:35.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131082025-11-12T20:51:35.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131092025-11-12T20:51:35.639ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
131102025-11-12T20:51:35.639ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
131112025-11-12T20:51:35.640ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a52fc06-cedb-458d-8099-9c2c8ee79858", "content-length": "81", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131122025-11-12T20:51:35.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131132025-11-12T20:51:35.640ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
131142025-11-12T20:51:35.640ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
131152025-11-12T20:51:35.641ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "15be1542-041c-44e4-9c6d-2c4347f94896", "content-length": "81", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131162025-11-12T20:51:35.643ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 364.337459ms iteration = 13 status = {"disabled":false,"errors":[],"sps":[]}
131172025-11-12T20:51:35.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
131182025-11-12T20:51:35.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131192025-11-12T20:51:35.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131202025-11-12T20:51:35.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131212025-11-12T20:51:35.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131222025-11-12T20:51:35.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131232025-11-12T20:51:35.654ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
131242025-11-12T20:51:35.654ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
131252025-11-12T20:51:35.654ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c2d96ee-0eec-487b-b055-b3f12ea3f2cf", "content-length": "26", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131262025-11-12T20:51:35.654ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
131272025-11-12T20:51:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
131282025-11-12T20:51:35.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131292025-11-12T20:51:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131302025-11-12T20:51:35.658ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
131312025-11-12T20:51:35.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131322025-11-12T20:51:35.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131332025-11-12T20:51:35.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131342025-11-12T20:51:35.661ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 382.378476ms iteration = 21 status = {"physical_disks_added":0}
131352025-11-12T20:51:35.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
131362025-11-12T20:51:35.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131372025-11-12T20:51:35.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131382025-11-12T20:51:35.667ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
131392025-11-12T20:51:35.668ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "3c20998f-1751-4885-8e37-7d1c0b610307", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131402025-11-12T20:51:35.668ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
131412025-11-12T20:51:35.670ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
131422025-11-12T20:51:35.670ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 377.975972ms iteration = 7 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
131432025-11-12T20:51:35.670ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
131442025-11-12T20:51:35.673ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 396.527981ms iteration = 13 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
131452025-11-12T20:51:35.681ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
131462025-11-12T20:51:35.681ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
131472025-11-12T20:51:35.681ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f7a69b0-d788-4425-a477-dde628894d27", "content-length": "26", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131482025-11-12T20:51:35.681ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
131492025-11-12T20:51:35.683ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
131502025-11-12T20:51:35.687ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = SledAgent step id = Ensure
131512025-11-12T20:51:35.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131522025-11-12T20:51:35.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131532025-11-12T20:51:35.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131542025-11-12T20:51:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131552025-11-12T20:51:35.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131562025-11-12T20:51:35.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
131572025-11-12T20:51:35.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131582025-11-12T20:51:35.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131592025-11-12T20:51:35.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131602025-11-12T20:51:35.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
131612025-11-12T20:51:35.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131622025-11-12T20:51:35.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131632025-11-12T20:51:35.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131642025-11-12T20:51:35.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131652025-11-12T20:51:35.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131662025-11-12T20:51:35.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131672025-11-12T20:51:35.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131682025-11-12T20:51:35.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
131692025-11-12T20:51:35.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131702025-11-12T20:51:35.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131712025-11-12T20:51:35.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131722025-11-12T20:51:35.728ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
131732025-11-12T20:51:35.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131742025-11-12T20:51:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131752025-11-12T20:51:35.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131762025-11-12T20:51:35.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131772025-11-12T20:51:35.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
131782025-11-12T20:51:35.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
131792025-11-12T20:51:35.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131802025-11-12T20:51:35.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
131812025-11-12T20:51:35.766ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
131822025-11-12T20:51:35.766ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
131832025-11-12T20:51:35.767ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e90c5f1f-664f-4c99-ba7d-48092f3ab454", "content-length": "372", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
131842025-11-12T20:51:35.767ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131852025-11-12T20:51:35.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131862025-11-12T20:51:35.768ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131872025-11-12T20:51:35.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
131882025-11-12T20:51:35.770ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131892025-11-12T20:51:35.773ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
131902025-11-12T20:51:35.773ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131912025-11-12T20:51:35.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
131922025-11-12T20:51:35.775ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131932025-11-12T20:51:35.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
131942025-11-12T20:51:35.777ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131952025-11-12T20:51:35.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
131962025-11-12T20:51:35.781ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
131972025-11-12T20:51:35.790ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
131982025-11-12T20:51:35.790ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7bdb32f3-d1d6-42ac-94cb-d7e134490616", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
131992025-11-12T20:51:35.790ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
132002025-11-12T20:51:35.792ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "77f76fa0-82d4-4702-99ab-20804129f3fe", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
132012025-11-12T20:51:35.792ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
132022025-11-12T20:51:35.794ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
132032025-11-12T20:51:35.794ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 517.709828ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
132042025-11-12T20:51:35.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132052025-11-12T20:51:35.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132062025-11-12T20:51:35.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132072025-11-12T20:51:35.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132082025-11-12T20:51:35.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132092025-11-12T20:51:35.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
132102025-11-12T20:51:35.817ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
132112025-11-12T20:51:35.817ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
132122025-11-12T20:51:35.817ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f864cf2e-7bd9-40b9-901b-b18eaf1bab89", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
132132025-11-12T20:51:35.817ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132142025-11-12T20:51:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132152025-11-12T20:51:35.820ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132162025-11-12T20:51:35.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132172025-11-12T20:51:35.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132182025-11-12T20:51:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132192025-11-12T20:51:35.826ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
132202025-11-12T20:51:35.826ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
132212025-11-12T20:51:35.827ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e630af0e-b726-4f0b-8d01-aa46703fc0c0", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
132222025-11-12T20:51:35.832ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 554.89434ms iteration = 13 status = {}
132232025-11-12T20:51:35.832ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 552.877568ms iteration = 19 status = {}
132242025-11-12T20:51:35.832ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Dependency
132252025-11-12T20:51:35.832ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
132262025-11-12T20:51:35.832ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132272025-11-12T20:51:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132282025-11-12T20:51:35.834ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132292025-11-12T20:51:35.836ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
132302025-11-12T20:51:35.836ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132312025-11-12T20:51:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132322025-11-12T20:51:35.837ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 332759783 } })
132332025-11-12T20:51:35.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132342025-11-12T20:51:35.839ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
132352025-11-12T20:51:35.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
132362025-11-12T20:51:35.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132372025-11-12T20:51:35.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132382025-11-12T20:51:35.841ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "a589ba60-e10d-4e2e-a5c9-e71db409eaec", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
132392025-11-12T20:51:35.843ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = FirewallRules step id = Ensure
132402025-11-12T20:51:35.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132412025-11-12T20:51:35.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132422025-11-12T20:51:35.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132432025-11-12T20:51:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132442025-11-12T20:51:35.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132452025-11-12T20:51:35.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132462025-11-12T20:51:35.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132472025-11-12T20:51:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132482025-11-12T20:51:35.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132492025-11-12T20:51:35.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132502025-11-12T20:51:35.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132512025-11-12T20:51:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132522025-11-12T20:51:35.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132532025-11-12T20:51:35.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132542025-11-12T20:51:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132552025-11-12T20:51:35.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132562025-11-12T20:51:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132572025-11-12T20:51:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132582025-11-12T20:51:35.861ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
132592025-11-12T20:51:35.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132602025-11-12T20:51:35.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132612025-11-12T20:51:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
132622025-11-12T20:51:35.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132632025-11-12T20:51:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132642025-11-12T20:51:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132652025-11-12T20:51:35.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132662025-11-12T20:51:35.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132672025-11-12T20:51:35.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132682025-11-12T20:51:35.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132692025-11-12T20:51:35.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132702025-11-12T20:51:35.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132712025-11-12T20:51:35.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132722025-11-12T20:51:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
132732025-11-12T20:51:35.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132742025-11-12T20:51:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132752025-11-12T20:51:35.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
132762025-11-12T20:51:35.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132772025-11-12T20:51:35.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132782025-11-12T20:51:35.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132792025-11-12T20:51:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132802025-11-12T20:51:35.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132812025-11-12T20:51:35.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132822025-11-12T20:51:35.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132832025-11-12T20:51:35.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132842025-11-12T20:51:35.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132852025-11-12T20:51:35.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132862025-11-12T20:51:35.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132872025-11-12T20:51:35.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
132882025-11-12T20:51:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132892025-11-12T20:51:35.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132902025-11-12T20:51:35.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132912025-11-12T20:51:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132922025-11-12T20:51:35.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132932025-11-12T20:51:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132942025-11-12T20:51:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132952025-11-12T20:51:35.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132962025-11-12T20:51:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132972025-11-12T20:51:35.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132982025-11-12T20:51:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132992025-11-12T20:51:35.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
133002025-11-12T20:51:35.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
133012025-11-12T20:51:35.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
133022025-11-12T20:51:35.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133032025-11-12T20:51:35.888ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
133042025-11-12T20:51:35.889ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc5334e2-5a95-4e84-a5e7-b02f71b821d0", "content-length": "691", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
133052025-11-12T20:51:35.889ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
133062025-11-12T20:51:35.889ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
133072025-11-12T20:51:35.889ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
133082025-11-12T20:51:35.890ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 611.742291ms iteration = 13 status = {}
133092025-11-12T20:51:35.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
133102025-11-12T20:51:35.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133112025-11-12T20:51:35.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133122025-11-12T20:51:35.893ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
133132025-11-12T20:51:35.894ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7819a16c-ee1d-4cba-ac12-e48bdf8108e1", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133142025-11-12T20:51:35.894ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133152025-11-12T20:51:35.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133162025-11-12T20:51:35.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133172025-11-12T20:51:35.896ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "fac556a0-3409-46bd-8c26-a4f02c1a92fa", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133182025-11-12T20:51:35.896ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133192025-11-12T20:51:35.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133202025-11-12T20:51:35.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133212025-11-12T20:51:35.898ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "32d9c506-0872-44fe-bcf7-b638be0dacb6", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133222025-11-12T20:51:35.898ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133232025-11-12T20:51:35.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133242025-11-12T20:51:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133252025-11-12T20:51:35.900ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
133262025-11-12T20:51:35.900ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 623.592917ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
133272025-11-12T20:51:35.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133282025-11-12T20:51:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133292025-11-12T20:51:35.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133302025-11-12T20:51:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133312025-11-12T20:51:35.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133322025-11-12T20:51:35.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133332025-11-12T20:51:35.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133342025-11-12T20:51:35.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
133352025-11-12T20:51:35.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133362025-11-12T20:51:35.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133372025-11-12T20:51:35.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133382025-11-12T20:51:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133392025-11-12T20:51:35.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133402025-11-12T20:51:35.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133412025-11-12T20:51:35.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133422025-11-12T20:51:35.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
133432025-11-12T20:51:35.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133442025-11-12T20:51:35.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133452025-11-12T20:51:35.933ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
133462025-11-12T20:51:35.933ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
133472025-11-12T20:51:35.933ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = bef546fb-7cfe-4e5f-8605-aa48c5dabc01 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
133482025-11-12T20:51:35.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
133492025-11-12T20:51:35.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
133502025-11-12T20:51:35.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133512025-11-12T20:51:35.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133522025-11-12T20:51:35.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133532025-11-12T20:51:35.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133542025-11-12T20:51:35.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133552025-11-12T20:51:35.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133562025-11-12T20:51:35.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
133572025-11-12T20:51:35.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
133582025-11-12T20:51:35.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133592025-11-12T20:51:35.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
133602025-11-12T20:51:35.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133612025-11-12T20:51:35.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133622025-11-12T20:51:35.942ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
133632025-11-12T20:51:35.942ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
133642025-11-12T20:51:35.942ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 663.021318ms iteration = 17 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
133652025-11-12T20:51:35.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133662025-11-12T20:51:35.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133672025-11-12T20:51:35.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133682025-11-12T20:51:35.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133692025-11-12T20:51:35.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133702025-11-12T20:51:35.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133712025-11-12T20:51:35.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133722025-11-12T20:51:35.950ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133732025-11-12T20:51:35.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133742025-11-12T20:51:35.952ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
133752025-11-12T20:51:35.952ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133762025-11-12T20:51:35.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133772025-11-12T20:51:35.952ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
133782025-11-12T20:51:35.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133792025-11-12T20:51:35.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133802025-11-12T20:51:35.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133812025-11-12T20:51:35.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133822025-11-12T20:51:35.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133832025-11-12T20:51:35.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133842025-11-12T20:51:35.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133852025-11-12T20:51:35.962ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
133862025-11-12T20:51:35.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133872025-11-12T20:51:35.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133882025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
133892025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
133902025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
133912025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
133922025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133932025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133942025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133952025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
133962025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
133972025-11-12T20:51:35.966ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
133982025-11-12T20:51:35.967ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
133992025-11-12T20:51:35.967ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134002025-11-12T20:51:35.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134012025-11-12T20:51:35.968ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
134022025-11-12T20:51:35.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134032025-11-12T20:51:35.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134042025-11-12T20:51:35.969ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
134052025-11-12T20:51:35.969ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
134062025-11-12T20:51:35.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134072025-11-12T20:51:35.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
134082025-11-12T20:51:35.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134092025-11-12T20:51:35.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134102025-11-12T20:51:35.977ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
134112025-11-12T20:51:35.977ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
134122025-11-12T20:51:35.977ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
134132025-11-12T20:51:35.977ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
134142025-11-12T20:51:35.977ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
134152025-11-12T20:51:35.983ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "22925da5-0915-4597-9c14-5fe86019bf49", "content-length": "372", "date": "Wed, 12 Nov 2025 20:51:35 GMT"} })
134162025-11-12T20:51:35.983ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134172025-11-12T20:51:35.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134182025-11-12T20:51:36.041ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
134192025-11-12T20:51:36.041ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
134202025-11-12T20:51:36.048ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4cd387a4-1bae-494f-b36d-074c97e8382b", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
134212025-11-12T20:51:36.048ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134222025-11-12T20:51:36.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134232025-11-12T20:51:36.050ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134242025-11-12T20:51:36.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134252025-11-12T20:51:36.053ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134262025-11-12T20:51:36.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134272025-11-12T20:51:36.056ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
134282025-11-12T20:51:36.056ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
134292025-11-12T20:51:36.056ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
134302025-11-12T20:51:36.056ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
134312025-11-12T20:51:36.057ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a73d4758-8eef-45c6-b12b-c4cd38f01e3f", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
134322025-11-12T20:51:36.058ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 226.734937ms iteration = 20 status = {}
134332025-11-12T20:51:36.089ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
134342025-11-12T20:51:36.089ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:40645
134352025-11-12T20:51:36.089ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:40645 req_id = 186837cb-dfd5-4473-ad67-c1c594060ffc uri = /metrics/producers version = 1.0.0
134362025-11-12T20:51:36.089ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:40645 req_id = 186837cb-dfd5-4473-ad67-c1c594060ffc roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
134372025-11-12T20:51:36.089ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:40645 req_id = 186837cb-dfd5-4473-ad67-c1c594060ffc resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
134382025-11-12T20:51:36.106ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
134392025-11-12T20:51:36.106ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
134402025-11-12T20:51:36.130ZINFOnexus: assigned collector to new producer collector_id = ecb0f932-e2e8-41d3-85a9-494e5cae5bce file = nexus/src/app/oximeter.rs:76 producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
134412025-11-12T20:51:36.130ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 40887 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:40645 req_id = 186837cb-dfd5-4473-ad67-c1c594060ffc response_code = 201 uri = /metrics/producers
134422025-11-12T20:51:36.130ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "186837cb-dfd5-4473-ad67-c1c594060ffc", "content-length": "41", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
134432025-11-12T20:51:36.131ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
134442025-11-12T20:51:36.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134452025-11-12T20:51:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134462025-11-12T20:51:36.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134472025-11-12T20:51:36.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134482025-11-12T20:51:36.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134492025-11-12T20:51:36.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
134502025-11-12T20:51:36.150ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
134512025-11-12T20:51:36.151ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "58b2596c-f54f-4130-b257-faf0349f9630", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
134522025-11-12T20:51:36.153ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
134532025-11-12T20:51:36.153ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = Dns step id = Ensure
134542025-11-12T20:51:36.153ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
134552025-11-12T20:51:36.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134562025-11-12T20:51:36.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134572025-11-12T20:51:36.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134582025-11-12T20:51:36.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134592025-11-12T20:51:36.155ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "60b3508c-0b45-4473-b7a5-7136772cbf1d", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
134602025-11-12T20:51:36.155ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
134612025-11-12T20:51:36.155ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
134622025-11-12T20:51:36.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134632025-11-12T20:51:36.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
134642025-11-12T20:51:36.160ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
134652025-11-12T20:51:36.160ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
134662025-11-12T20:51:36.160ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
134672025-11-12T20:51:36.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134682025-11-12T20:51:36.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134692025-11-12T20:51:36.161ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9791be48-bb93-454c-9441-007b2bfcbb37", "content-length": "682", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
134702025-11-12T20:51:36.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134712025-11-12T20:51:36.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134722025-11-12T20:51:36.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134732025-11-12T20:51:36.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134742025-11-12T20:51:36.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134752025-11-12T20:51:36.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
134762025-11-12T20:51:36.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134772025-11-12T20:51:36.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134782025-11-12T20:51:36.168ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
134792025-11-12T20:51:36.168ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
134802025-11-12T20:51:36.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134812025-11-12T20:51:36.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134822025-11-12T20:51:36.168ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134832025-11-12T20:51:36.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134842025-11-12T20:51:36.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134852025-11-12T20:51:36.170ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134862025-11-12T20:51:36.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134872025-11-12T20:51:36.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
134882025-11-12T20:51:36.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134892025-11-12T20:51:36.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134902025-11-12T20:51:36.176ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
134912025-11-12T20:51:36.176ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
134922025-11-12T20:51:36.176ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
134932025-11-12T20:51:36.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134942025-11-12T20:51:36.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134952025-11-12T20:51:36.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134962025-11-12T20:51:36.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134972025-11-12T20:51:36.178ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 898.939442ms iteration = 13 status = {}
134982025-11-12T20:51:36.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134992025-11-12T20:51:36.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
135002025-11-12T20:51:36.183ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
135012025-11-12T20:51:36.183ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
135022025-11-12T20:51:36.183ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
135032025-11-12T20:51:36.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135042025-11-12T20:51:36.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135052025-11-12T20:51:36.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135062025-11-12T20:51:36.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
135072025-11-12T20:51:36.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135082025-11-12T20:51:36.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135092025-11-12T20:51:36.230ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
135102025-11-12T20:51:36.230ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
135112025-11-12T20:51:36.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135122025-11-12T20:51:36.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135132025-11-12T20:51:36.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135142025-11-12T20:51:36.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
135152025-11-12T20:51:36.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135162025-11-12T20:51:36.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135172025-11-12T20:51:36.242ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
135182025-11-12T20:51:36.242ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
135192025-11-12T20:51:36.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135202025-11-12T20:51:36.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135212025-11-12T20:51:36.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135222025-11-12T20:51:36.246ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
135232025-11-12T20:51:36.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135242025-11-12T20:51:36.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135252025-11-12T20:51:36.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135262025-11-12T20:51:36.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135272025-11-12T20:51:36.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135282025-11-12T20:51:36.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135292025-11-12T20:51:36.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135302025-11-12T20:51:36.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
135312025-11-12T20:51:36.256ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
135322025-11-12T20:51:36.256ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
135332025-11-12T20:51:36.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = OmicronZones step id = Cleanup
135342025-11-12T20:51:36.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = OmicronZones step id = Cleanup
135352025-11-12T20:51:36.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = PhysicalDisks step id = Cleanup
135362025-11-12T20:51:36.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = Clickhouse step id = Ensure
135372025-11-12T20:51:36.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = Clickhouse step id = Ensure
135382025-11-12T20:51:36.291ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
135392025-11-12T20:51:36.295ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "55ec6f4d-1071-42be-bd23-2621214838fc", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
135402025-11-12T20:51:36.296ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = SupportBundles step id = Cleanup
135412025-11-12T20:51:36.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135422025-11-12T20:51:36.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135432025-11-12T20:51:36.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135442025-11-12T20:51:36.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
135452025-11-12T20:51:36.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135462025-11-12T20:51:36.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135472025-11-12T20:51:36.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135482025-11-12T20:51:36.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135492025-11-12T20:51:36.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135502025-11-12T20:51:36.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
135512025-11-12T20:51:36.347ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = OmicronZones step id = Cleanup
135522025-11-12T20:51:36.347ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
135532025-11-12T20:51:36.347ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
135542025-11-12T20:51:36.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135552025-11-12T20:51:36.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135562025-11-12T20:51:36.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135572025-11-12T20:51:36.351ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
135582025-11-12T20:51:36.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135592025-11-12T20:51:36.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135602025-11-12T20:51:36.396ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
135612025-11-12T20:51:36.396ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
135622025-11-12T20:51:36.396ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = Cockroach step id = Ensure
135632025-11-12T20:51:36.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135642025-11-12T20:51:36.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135652025-11-12T20:51:36.412ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
135662025-11-12T20:51:36.412ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4d1704a9-7b57-4d60-8253-937d79f18c46 step component = MgsUpdates step id = Ensure
135672025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Dependency
135682025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Dependency
135692025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135702025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
135712025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
135722025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.136352927s iteration = 8 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3840,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":139533701,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":139533661,"secs":0}},"event_index":1,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":139558101,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":186774153,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":186774103,"secs":0}},"event_index":2,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":326345635,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":84211780,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":84211740,"secs":0}},"event_index":3,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":410573815,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":155369360,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":155369310,"secs":0}},"event_index":4,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":565975726,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":310091454,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":310091404,"secs":0}},"event_index":5,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":876078781,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":103269413,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":103269382,"secs":0}},"event_index":6,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979362864,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5670,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5630,"secs":0}},"event_index":7,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979379254,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2380,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2340,"secs":0}},"event_index":8,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979383844,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2700,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2660,"secs":0}},"event_index":9,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979388214,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1880,"secs":0}},"event_index":10,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979391504,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40532479,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":40532439,"secs":0}},"event_index":11,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":19925123,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50833165,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50833095,"secs":0}},"event_index":12,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":70770929,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49094010,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":49093950,"secs":0}},"event_index":13,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":119881580,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16053496,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16053436,"secs":0}},"event_index":14,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":135948156,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4430,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4390,"secs":0}},"event_index":15,"execution_id":"46ae391f-b18b-40a7-aebb-d984825fc5ba","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":135960226,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
135732025-11-12T20:51:36.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
135742025-11-12T20:51:36.414ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135752025-11-12T20:51:36.415ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135762025-11-12T20:51:36.417ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
135772025-11-12T20:51:36.417ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135782025-11-12T20:51:36.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
135792025-11-12T20:51:36.418ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
135802025-11-12T20:51:36.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
135812025-11-12T20:51:36.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
135822025-11-12T20:51:36.453ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
135832025-11-12T20:51:36.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
135842025-11-12T20:51:36.455ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135852025-11-12T20:51:36.457ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
135862025-11-12T20:51:36.458ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
135872025-11-12T20:51:36.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
135882025-11-12T20:51:36.460ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
135892025-11-12T20:51:36.460ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 47.038568ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
135902025-11-12T20:51:36.475ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(0f5ca54a-65d9-456c-b99e-32ccac684146, 0), (bc3618b1-e97a-4c0f-9bea-49c834581f18, 0), (53a0c9f3-40f8-426b-999e-1a06cd803cef, 0), (a6ce4867-8d96-4d76-9bcd-36258ef2dd87, 0)] collection_id = 0f5ca54a-65d9-456c-b99e-32ccac684146
135912025-11-12T20:51:36.475ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135922025-11-12T20:51:36.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
135932025-11-12T20:51:36.617ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 13 transaction = inventory_delete_collection
135942025-11-12T20:51:36.665ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
135952025-11-12T20:51:36.665ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 0f5ca54a-65d9-456c-b99e-32ccac684146 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
135962025-11-12T20:51:36.665ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135972025-11-12T20:51:36.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
135982025-11-12T20:51:36.668ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(bc3618b1-e97a-4c0f-9bea-49c834581f18, 0), (53a0c9f3-40f8-426b-999e-1a06cd803cef, 0), (a6ce4867-8d96-4d76-9bcd-36258ef2dd87, 0)]
135992025-11-12T20:51:36.669ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1292, tv_nsec: 164046575 } })
136002025-11-12T20:51:36.705ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1292, tv_nsec: 200336834 } })
136012025-11-12T20:51:36.705ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
136022025-11-12T20:51:36.705ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
136032025-11-12T20:51:36.705ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
136042025-11-12T20:51:36.706ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "88795c59-6cbb-4a7f-b9e9-4817dac93c4d", "content-length": "476", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136052025-11-12T20:51:36.706ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
136062025-11-12T20:51:36.706ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f527e2f-e17d-4f1a-bef3-2edc691aa66f", "content-length": "734", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136072025-11-12T20:51:36.706ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
136082025-11-12T20:51:36.707ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "302e807f-5416-4945-8f86-dec54518220b", "content-length": "111", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136092025-11-12T20:51:36.707ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
136102025-11-12T20:51:36.707ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ada61d4-e087-450f-8734-59f6893570fd", "content-length": "111", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136112025-11-12T20:51:36.707ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
136122025-11-12T20:51:36.708ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "98c55a4f-6713-4b36-a3a9-c34c31b9f520", "content-length": "163", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136132025-11-12T20:51:36.708ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
136142025-11-12T20:51:36.708ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a84b88e-b196-4a71-bbc2-239be720d520", "content-length": "163", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136152025-11-12T20:51:36.708ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
136162025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "673560fb-8974-43c7-9673-ef283f769df9", "content-length": "166", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136172025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
136182025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ea4cd24-76e9-4306-a724-118a99862d3a", "content-length": "166", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136192025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
136202025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77a5e0f6-0a1e-4167-9dd7-31f6a588852b", "content-length": "702", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136212025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
136222025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "858180f2-2866-4762-9213-60fd4c9ad849", "content-length": "727", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136232025-11-12T20:51:36.709ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
136242025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55cd414d-6288-4de0-be90-d324c261f9cc", "content-length": "729", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136252025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
136262025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3922030-8f4e-4ff1-873a-b337da53a905", "content-length": "728", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136272025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
136282025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1377723-ffbb-40a3-8963-68de61c7bcba", "content-length": "722", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136292025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
136302025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "ece23191-35ff-4e4f-b47f-15e0f2b6669e", "content-length": "10", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136312025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
136322025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "580df719-60b0-423e-9b58-a3ec526198e1", "content-length": "144", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136332025-11-12T20:51:36.710ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
136342025-11-12T20:51:36.711ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6197cba-3391-49c4-9bab-c64c10059a99", "content-length": "147", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136352025-11-12T20:51:36.711ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
136362025-11-12T20:51:36.711ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0913794f-6eea-4aed-a63a-2ab36f24d296", "content-length": "109", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136372025-11-12T20:51:36.711ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
136382025-11-12T20:51:36.711ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a9bf918-47f6-45b8-8d35-1a25177b89bf", "content-length": "109", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136392025-11-12T20:51:36.711ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
136402025-11-12T20:51:36.712ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e531c950-4548-4923-b5c1-be323110a835", "content-length": "163", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136412025-11-12T20:51:36.712ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
136422025-11-12T20:51:36.712ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ad2c6b8-8286-4385-9335-ded7aacd8856", "content-length": "163", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136432025-11-12T20:51:36.712ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
136442025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e087ebc0-0fdf-46a9-8509-7e6cdd32357b", "content-length": "166", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136452025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
136462025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "41e00d1e-5265-4e08-a1f8-f0cc94330fbc", "content-length": "166", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136472025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
136482025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4380b5a-ec68-40c0-b95a-6ef627225030", "content-length": "702", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136492025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
136502025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "afc15a9a-4ac0-4b06-b793-3aefa055f500", "content-length": "727", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136512025-11-12T20:51:36.713ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
136522025-11-12T20:51:36.714ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ad2b94b-e80a-4a44-a3e2-583da920d38c", "content-length": "729", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136532025-11-12T20:51:36.714ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
136542025-11-12T20:51:36.714ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9cc9aa1-48b3-4de0-b6fc-90d3ba8e0094", "content-length": "728", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136552025-11-12T20:51:36.714ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136562025-11-12T20:51:36.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
136572025-11-12T20:51:36.716ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136582025-11-12T20:51:36.718ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
136592025-11-12T20:51:36.718ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136602025-11-12T20:51:36.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
136612025-11-12T20:51:36.719ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136622025-11-12T20:51:36.721ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
136632025-11-12T20:51:36.721ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136642025-11-12T20:51:36.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
136652025-11-12T20:51:36.831ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
136662025-11-12T20:51:36.831ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
136672025-11-12T20:51:36.881ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8076da5e-6fd8-41d1-8389-04af4bc5126f", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136682025-11-12T20:51:36.883ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
136692025-11-12T20:51:36.883ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
136702025-11-12T20:51:36.883ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
136712025-11-12T20:51:36.883ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
136722025-11-12T20:51:36.883ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
136732025-11-12T20:51:36.883ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
136742025-11-12T20:51:36.883ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
136752025-11-12T20:51:36.883ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
136762025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e04896ba-adaf-4876-bb1d-4a715bc13598", "content-length": "64", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136772025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "797436ea-44ef-416b-bab8-4886172666aa", "content-length": "64", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136782025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
136792025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
136802025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b61b8e99-d54e-4db0-9ae0-44bb1bc5b214", "content-length": "64", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136812025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7865d67e-6703-44b8-a1d1-c11d2bea8240", "content-length": "64", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136822025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c267b750-0d5a-48af-b014-d516147555b8", "content-length": "64", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136832025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
136842025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
136852025-11-12T20:51:36.884ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
136862025-11-12T20:51:36.924ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "72022d7e-d039-4df2-b32f-cc4285a6b8f4", "content-length": "426450", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136872025-11-12T20:51:36.924ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ea3ef1d-a766-445c-8b13-6ff5a60ea11c", "content-length": "396115", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136882025-11-12T20:51:36.924ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "770804e6-ca5e-423e-ab20-d78e2892d774", "content-length": "398850", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136892025-11-12T20:51:36.924ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "36721683-d18f-4396-86cd-9f65575f555c", "content-length": "394057", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136902025-11-12T20:51:36.926ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "987d91fe-25b2-4d00-abf3-d9f044801ae6", "content-length": "396824", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136912025-11-12T20:51:36.929ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
136922025-11-12T20:51:36.929ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
136932025-11-12T20:51:36.929ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
136942025-11-12T20:51:36.929ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
136952025-11-12T20:51:36.929ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
136962025-11-12T20:51:36.960ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
136972025-11-12T20:51:36.960ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
136982025-11-12T20:51:36.963ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "1380cd31-15b0-4e8a-b13e-1c5e73ede034", "content-length": "128", "date": "Wed, 12 Nov 2025 20:51:36 GMT"} })
136992025-11-12T20:51:36.965ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
137002025-11-12T20:51:37.057ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
137012025-11-12T20:51:37.057ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
137022025-11-12T20:51:37.058ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "93841731-3cc2-4715-98c7-e76794acacf8", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:51:37 GMT"} })
137032025-11-12T20:51:37.058ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
137042025-11-12T20:51:37.061ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
137052025-11-12T20:51:37.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
137062025-11-12T20:51:37.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "59550c9c-1a78-44c6-b1ca-d08f5c6a8394", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:51:37 GMT"} })
137072025-11-12T20:51:37.061ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
137082025-11-12T20:51:37.062ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
137092025-11-12T20:51:37.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
137102025-11-12T20:51:37.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "049c9a38-2c88-4a39-b7b3-eb7e8aaf711a", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:51:37 GMT"} })
137112025-11-12T20:51:37.063ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
137122025-11-12T20:51:37.065ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
137132025-11-12T20:51:37.065ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
137142025-11-12T20:51:37.067ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
137152025-11-12T20:51:37.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
137162025-11-12T20:51:37.069ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137172025-11-12T20:51:37.072ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
137182025-11-12T20:51:37.072ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
137192025-11-12T20:51:37.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
137202025-11-12T20:51:37.267ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb file = nexus/db-queries/src/db/datastore/inventory.rs:1614
137212025-11-12T20:51:37.267ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb time_started = 2025-11-12 20:51:36.705866 UTC
137222025-11-12T20:51:37.267ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 33 reason = Dependency
137232025-11-12T20:51:37.267ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
137242025-11-12T20:51:37.267ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 854.081412ms iteration = 8 status = {"collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","time_done":"2025-11-12 20:51:37.065681 UTC","time_started":"2025-11-12 20:51:36.705866 UTC"}
137252025-11-12T20:51:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
137262025-11-12T20:51:37.269ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137272025-11-12T20:51:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
137282025-11-12T20:51:37.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
137292025-11-12T20:51:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
137302025-11-12T20:51:37.274ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
137312025-11-12T20:51:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
137322025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 53.619458ms iteration = 33 status = {"Loaded":{"collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","time_loaded":"2025-11-12T20:51:37.267134398Z","time_started":"2025-11-12T20:51:36.705866Z"}}
137332025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 11 reason = Dependency
137342025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 18 reason = Dependency
137352025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Dependency
137362025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137372025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137382025-11-12T20:51:37.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137392025-11-12T20:51:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137402025-11-12T20:51:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137412025-11-12T20:51:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137422025-11-12T20:51:37.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137432025-11-12T20:51:37.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137442025-11-12T20:51:37.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137452025-11-12T20:51:37.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
137462025-11-12T20:51:37.325ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137472025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137482025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
137492025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137502025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
137512025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137522025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137532025-11-12T20:51:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137542025-11-12T20:51:37.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137552025-11-12T20:51:37.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137562025-11-12T20:51:37.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137572025-11-12T20:51:37.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137582025-11-12T20:51:37.329ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137592025-11-12T20:51:37.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
137602025-11-12T20:51:37.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137612025-11-12T20:51:37.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137622025-11-12T20:51:37.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
137632025-11-12T20:51:37.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137642025-11-12T20:51:37.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137652025-11-12T20:51:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
137662025-11-12T20:51:37.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137672025-11-12T20:51:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137682025-11-12T20:51:37.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137692025-11-12T20:51:37.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137702025-11-12T20:51:37.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137712025-11-12T20:51:37.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137722025-11-12T20:51:37.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137732025-11-12T20:51:37.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137742025-11-12T20:51:37.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
137752025-11-12T20:51:37.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137762025-11-12T20:51:37.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
137772025-11-12T20:51:37.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137782025-11-12T20:51:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137792025-11-12T20:51:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137802025-11-12T20:51:37.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137812025-11-12T20:51:37.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137822025-11-12T20:51:37.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
137832025-11-12T20:51:37.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137842025-11-12T20:51:37.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137852025-11-12T20:51:37.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
137862025-11-12T20:51:37.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137872025-11-12T20:51:37.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137882025-11-12T20:51:37.350ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
137892025-11-12T20:51:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137902025-11-12T20:51:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137912025-11-12T20:51:37.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137922025-11-12T20:51:37.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
137932025-11-12T20:51:37.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137942025-11-12T20:51:37.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137952025-11-12T20:51:37.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137962025-11-12T20:51:37.358ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
137972025-11-12T20:51:37.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
137982025-11-12T20:51:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
137992025-11-12T20:51:37.360ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
138002025-11-12T20:51:37.360ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 39.579435ms iteration = 11 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
138012025-11-12T20:51:37.381ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 60.726891ms iteration = 22 status = {"physical_disks_added":0}
138022025-11-12T20:51:37.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138032025-11-12T20:51:37.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138042025-11-12T20:51:37.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138052025-11-12T20:51:37.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138062025-11-12T20:51:37.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138072025-11-12T20:51:37.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138082025-11-12T20:51:37.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138092025-11-12T20:51:37.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138102025-11-12T20:51:37.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138112025-11-12T20:51:37.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138122025-11-12T20:51:37.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138132025-11-12T20:51:37.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
138142025-11-12T20:51:37.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138152025-11-12T20:51:37.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138162025-11-12T20:51:37.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138172025-11-12T20:51:37.405ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138182025-11-12T20:51:37.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138192025-11-12T20:51:37.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138202025-11-12T20:51:37.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138212025-11-12T20:51:37.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138222025-11-12T20:51:37.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138232025-11-12T20:51:37.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
138242025-11-12T20:51:37.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138252025-11-12T20:51:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138262025-11-12T20:51:37.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138272025-11-12T20:51:37.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138282025-11-12T20:51:37.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138292025-11-12T20:51:37.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
138302025-11-12T20:51:37.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138312025-11-12T20:51:37.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138322025-11-12T20:51:37.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138332025-11-12T20:51:37.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138342025-11-12T20:51:37.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138352025-11-12T20:51:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
138362025-11-12T20:51:37.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138372025-11-12T20:51:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138382025-11-12T20:51:37.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138392025-11-12T20:51:37.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138402025-11-12T20:51:37.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138412025-11-12T20:51:37.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138422025-11-12T20:51:37.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138432025-11-12T20:51:37.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138442025-11-12T20:51:37.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138452025-11-12T20:51:37.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138462025-11-12T20:51:37.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138472025-11-12T20:51:37.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138482025-11-12T20:51:37.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138492025-11-12T20:51:37.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138502025-11-12T20:51:37.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138512025-11-12T20:51:37.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138522025-11-12T20:51:37.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138532025-11-12T20:51:37.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138542025-11-12T20:51:37.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138552025-11-12T20:51:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138562025-11-12T20:51:37.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138572025-11-12T20:51:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138582025-11-12T20:51:37.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138592025-11-12T20:51:37.464ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138602025-11-12T20:51:37.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138612025-11-12T20:51:37.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138622025-11-12T20:51:37.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138632025-11-12T20:51:37.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138642025-11-12T20:51:37.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138652025-11-12T20:51:37.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138662025-11-12T20:51:37.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138672025-11-12T20:51:37.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
138682025-11-12T20:51:37.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138692025-11-12T20:51:37.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138702025-11-12T20:51:37.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138712025-11-12T20:51:37.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138722025-11-12T20:51:37.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138732025-11-12T20:51:37.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
138742025-11-12T20:51:37.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138752025-11-12T20:51:37.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138762025-11-12T20:51:37.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138772025-11-12T20:51:37.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138782025-11-12T20:51:37.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138792025-11-12T20:51:37.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
138802025-11-12T20:51:37.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138812025-11-12T20:51:37.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138822025-11-12T20:51:37.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138832025-11-12T20:51:37.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138842025-11-12T20:51:37.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138852025-11-12T20:51:37.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
138862025-11-12T20:51:37.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138872025-11-12T20:51:37.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138882025-11-12T20:51:37.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138892025-11-12T20:51:37.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138902025-11-12T20:51:37.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138912025-11-12T20:51:37.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
138922025-11-12T20:51:37.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138932025-11-12T20:51:37.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138942025-11-12T20:51:37.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138952025-11-12T20:51:37.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138962025-11-12T20:51:37.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138972025-11-12T20:51:37.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
138982025-11-12T20:51:37.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138992025-11-12T20:51:37.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139002025-11-12T20:51:37.579ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
139012025-11-12T20:51:37.579ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
139022025-11-12T20:51:37.579ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a9fd2871-6b44-4043-b64a-700fc1e42674 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
139032025-11-12T20:51:37.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139042025-11-12T20:51:37.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139052025-11-12T20:51:37.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139062025-11-12T20:51:37.584ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
139072025-11-12T20:51:37.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139082025-11-12T20:51:37.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139092025-11-12T20:51:37.587ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
139102025-11-12T20:51:37.587ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
139112025-11-12T20:51:37.587ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 267.177125ms iteration = 18 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
139122025-11-12T20:51:37.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 297 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = b82e45fe-4bca-47ca-ae72-5a2ce961220b response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
139132025-11-12T20:51:40.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 74 reason = Timeout
139142025-11-12T20:51:40.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
139152025-11-12T20:51:40.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
139162025-11-12T20:51:40.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139172025-11-12T20:51:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
139182025-11-12T20:51:40.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
139192025-11-12T20:51:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
139202025-11-12T20:51:40.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
139212025-11-12T20:51:40.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.427852ms iteration = 74 status = {"config_updated":false}
139222025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
139232025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
139242025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
139252025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139262025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 75 reason = Timeout
139272025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
139282025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
139292025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139302025-11-12T20:51:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
139312025-11-12T20:51:45.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139322025-11-12T20:51:45.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139332025-11-12T20:51:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
139342025-11-12T20:51:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139352025-11-12T20:51:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139362025-11-12T20:51:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
139372025-11-12T20:51:45.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
139382025-11-12T20:51:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
139392025-11-12T20:51:45.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139402025-11-12T20:51:45.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139412025-11-12T20:51:45.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
139422025-11-12T20:51:45.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.592082ms iteration = 75 status = {"config_updated":false}
139432025-11-12T20:51:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139442025-11-12T20:51:45.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
139452025-11-12T20:51:45.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139462025-11-12T20:51:45.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139472025-11-12T20:51:45.307ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
139482025-11-12T20:51:45.307ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.275781ms iteration = 38 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
139492025-11-12T20:51:45.313ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
139502025-11-12T20:51:45.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
139512025-11-12T20:51:45.350ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.658274ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
139522025-11-12T20:51:47.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 383 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = ced3f68d-d356-4945-bc7d-9c21642570f1 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
139532025-11-12T20:51:50.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 34 reason = Timeout
139542025-11-12T20:51:50.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
139552025-11-12T20:51:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
139562025-11-12T20:51:50.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 76 reason = Timeout
139572025-11-12T20:51:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
139582025-11-12T20:51:50.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
139592025-11-12T20:51:50.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139602025-11-12T20:51:50.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139612025-11-12T20:51:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
139622025-11-12T20:51:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
139632025-11-12T20:51:50.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
139642025-11-12T20:51:50.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
139652025-11-12T20:51:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
139662025-11-12T20:51:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
139672025-11-12T20:51:50.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
139682025-11-12T20:51:50.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.513479ms iteration = 76 status = {"config_updated":false}
139692025-11-12T20:51:50.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb original_time_started = 2025-11-12 20:51:36.705866 UTC
139702025-11-12T20:51:50.286ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.463574ms iteration = 34 status = {"Loaded":{"collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","time_loaded":"2025-11-12T20:51:50.278557451Z","time_started":"2025-11-12T20:51:36.705866Z"}}
139712025-11-12T20:51:50.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 26 reason = Timeout
139722025-11-12T20:51:50.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
139732025-11-12T20:51:50.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
139742025-11-12T20:51:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139752025-11-12T20:51:50.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
139762025-11-12T20:51:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
139772025-11-12T20:51:50.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
139782025-11-12T20:51:50.291ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.42909ms iteration = 26 status = "NoSitrep"
139792025-11-12T20:51:50.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 26 reason = Timeout
139802025-11-12T20:51:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
139812025-11-12T20:51:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
139822025-11-12T20:51:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
139832025-11-12T20:51:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
139842025-11-12T20:51:50.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139852025-11-12T20:51:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
139862025-11-12T20:51:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
139872025-11-12T20:51:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
139882025-11-12T20:51:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
139892025-11-12T20:51:50.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
139902025-11-12T20:51:50.303ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139912025-11-12T20:51:50.304ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
139922025-11-12T20:51:50.305ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
139932025-11-12T20:51:50.305ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.09937ms iteration = 26 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
139942025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
139952025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
139962025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139972025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 77 reason = Timeout
139982025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
139992025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
140002025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
140012025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
140022025-11-12T20:51:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
140032025-11-12T20:51:55.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140042025-11-12T20:51:55.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140052025-11-12T20:51:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
140062025-11-12T20:51:55.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
140072025-11-12T20:51:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
140082025-11-12T20:51:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
140092025-11-12T20:51:55.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
140102025-11-12T20:51:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
140112025-11-12T20:51:55.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
140122025-11-12T20:51:55.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.414446ms iteration = 77 status = {"config_updated":false}
140132025-11-12T20:51:55.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
140142025-11-12T20:51:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
140152025-11-12T20:51:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140162025-11-12T20:51:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
140172025-11-12T20:51:55.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
140182025-11-12T20:51:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
140192025-11-12T20:51:55.310ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
140202025-11-12T20:51:55.310ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.598221ms iteration = 39 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
140212025-11-12T20:51:55.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
140222025-11-12T20:51:55.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
140232025-11-12T20:51:55.321ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.671509ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
140242025-11-12T20:51:57.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 434 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 02ac2a99-c92a-40eb-b839-efa5c24f890f response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
140252025-11-12T20:52:00.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 78 reason = Timeout
140262025-11-12T20:52:00.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
140272025-11-12T20:52:00.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
140282025-11-12T20:52:00.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140292025-11-12T20:52:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
140302025-11-12T20:52:00.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
140312025-11-12T20:52:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
140322025-11-12T20:52:00.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
140332025-11-12T20:52:00.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.33416ms iteration = 78 status = {"config_updated":false}
140342025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
140352025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 14 reason = Timeout
140362025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
140372025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
140382025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
140392025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
140402025-11-12T20:52:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
140412025-11-12T20:52:05.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 35 reason = Timeout
140422025-11-12T20:52:05.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
140432025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
140442025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
140452025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
140462025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
140472025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
140482025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140492025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
140502025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
140512025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
140522025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
140532025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
140542025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
140552025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
140562025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
140572025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
140582025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
140592025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
140602025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
140612025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
140622025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
140632025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
140642025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
140652025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
140662025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 79 reason = Timeout
140672025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
140682025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
140692025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
140702025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
140712025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
140722025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
140732025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
140742025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
140752025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
140762025-11-12T20:52:05.279ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
140772025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
140782025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 14 reason = Timeout
140792025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
140802025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
140812025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
140822025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
140832025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
140842025-11-12T20:52:05.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1320, tv_nsec: 775143769 } })
140852025-11-12T20:52:05.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1320, tv_nsec: 775167790 } })
140862025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
140872025-11-12T20:52:05.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1320, tv_nsec: 775243221 } })
140882025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
140892025-11-12T20:52:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
140902025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
140912025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
140922025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
140932025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
140942025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
140952025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
140962025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
140972025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
140982025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
140992025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
141002025-11-12T20:52:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
141012025-11-12T20:52:05.280ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
141022025-11-12T20:52:05.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141032025-11-12T20:52:05.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141042025-11-12T20:52:05.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141052025-11-12T20:52:05.283ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.490424ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
141062025-11-12T20:52:05.284ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
141072025-11-12T20:52:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
141082025-11-12T20:52:05.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 27 reason = Timeout
141092025-11-12T20:52:05.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
141102025-11-12T20:52:05.287ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141112025-11-12T20:52:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
141122025-11-12T20:52:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
141132025-11-12T20:52:05.288ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
141142025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
141152025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
141162025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
141172025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
141182025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
141192025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
141202025-11-12T20:52:05.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141212025-11-12T20:52:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
141222025-11-12T20:52:05.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 27 reason = Timeout
141232025-11-12T20:52:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
141242025-11-12T20:52:05.292ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
141252025-11-12T20:52:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.77844ms iteration = 79 status = {"config_updated":false}
141262025-11-12T20:52:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
141272025-11-12T20:52:05.292ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb original_time_started = 2025-11-12 20:51:36.705866 UTC
141282025-11-12T20:52:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.92714ms iteration = 35 status = {"Loaded":{"collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","time_loaded":"2025-11-12T20:52:05.278730134Z","time_started":"2025-11-12T20:51:36.705866Z"}}
141292025-11-12T20:52:05.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
141302025-11-12T20:52:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
141312025-11-12T20:52:05.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
141322025-11-12T20:52:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
141332025-11-12T20:52:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
141342025-11-12T20:52:05.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
141352025-11-12T20:52:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
141362025-11-12T20:52:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
141372025-11-12T20:52:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141382025-11-12T20:52:05.296ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.957479ms iteration = 14 status = 0
141392025-11-12T20:52:05.297ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.43227ms iteration = 27 status = "NoSitrep"
141402025-11-12T20:52:05.298ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
141412025-11-12T20:52:05.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
141422025-11-12T20:52:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
141432025-11-12T20:52:05.304ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
141442025-11-12T20:52:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
141452025-11-12T20:52:05.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
141462025-11-12T20:52:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
141472025-11-12T20:52:05.355ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141482025-11-12T20:52:05.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141492025-11-12T20:52:05.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
141502025-11-12T20:52:05.356ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
141512025-11-12T20:52:05.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
141522025-11-12T20:52:05.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
141532025-11-12T20:52:05.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 77.804434ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
141542025-11-12T20:52:05.357ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141552025-11-12T20:52:05.357ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 78.43981ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
141562025-11-12T20:52:05.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141572025-11-12T20:52:05.358ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
141582025-11-12T20:52:05.358ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
141592025-11-12T20:52:05.358ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
141602025-11-12T20:52:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
141612025-11-12T20:52:05.360ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
141622025-11-12T20:52:05.360ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
141632025-11-12T20:52:05.360ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141642025-11-12T20:52:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
141652025-11-12T20:52:05.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141662025-11-12T20:52:05.360ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141672025-11-12T20:52:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
141682025-11-12T20:52:05.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
141692025-11-12T20:52:05.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.820688ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
141702025-11-12T20:52:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
141712025-11-12T20:52:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
141722025-11-12T20:52:05.361ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
141732025-11-12T20:52:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
141742025-11-12T20:52:05.362ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141752025-11-12T20:52:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
141762025-11-12T20:52:05.362ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
141772025-11-12T20:52:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
141782025-11-12T20:52:05.363ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
141792025-11-12T20:52:05.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
141802025-11-12T20:52:05.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
141812025-11-12T20:52:05.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
141822025-11-12T20:52:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
141832025-11-12T20:52:05.364ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
141842025-11-12T20:52:05.364ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141852025-11-12T20:52:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
141862025-11-12T20:52:05.365ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
141872025-11-12T20:52:05.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141882025-11-12T20:52:05.365ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 86.160039ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
141892025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
141902025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141912025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
141922025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
141932025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
141942025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141952025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
141962025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
141972025-11-12T20:52:05.367ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
141982025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "78678338-d785-4f88-8d49-d7e8a1c23c86", "content-length": "78", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
141992025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
142002025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
142012025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
142022025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
142032025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
142042025-11-12T20:52:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
142052025-11-12T20:52:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
142062025-11-12T20:52:05.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142072025-11-12T20:52:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
142082025-11-12T20:52:05.370ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1320, tv_nsec: 865384954 } })
142092025-11-12T20:52:05.370ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
142102025-11-12T20:52:05.370ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
142112025-11-12T20:52:05.371ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
142122025-11-12T20:52:05.371ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
142132025-11-12T20:52:05.372ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcff40c7-4f10-4ddd-abc7-1859bb8f4918", "content-length": "26", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142142025-11-12T20:52:05.372ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
142152025-11-12T20:52:05.374ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
142162025-11-12T20:52:05.375ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
142172025-11-12T20:52:05.375ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
142182025-11-12T20:52:05.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
142192025-11-12T20:52:05.382ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
142202025-11-12T20:52:05.382ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
142212025-11-12T20:52:05.384ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "38fb7d60-8332-4ae7-9f8c-c67d5554c104", "content-length": "81", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142222025-11-12T20:52:05.395ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142232025-11-12T20:52:05.397ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
142242025-11-12T20:52:05.398ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
142252025-11-12T20:52:05.398ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 106.814634ms iteration = 27 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
142262025-11-12T20:52:05.399ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
142272025-11-12T20:52:05.399ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 120.57983ms iteration = 40 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
142282025-11-12T20:52:05.413ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 134.274715ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
142292025-11-12T20:52:05.419ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
142302025-11-12T20:52:05.419ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 140.309681ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
142312025-11-12T20:52:05.420ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 141.722498ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
142322025-11-12T20:52:05.421ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 145.538557ms iteration = 14 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
142332025-11-12T20:52:05.422ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
142342025-11-12T20:52:05.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
142352025-11-12T20:52:05.426ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142362025-11-12T20:52:05.429ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
142372025-11-12T20:52:05.429ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
142382025-11-12T20:52:05.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
142392025-11-12T20:52:05.431ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
142402025-11-12T20:52:05.431ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
142412025-11-12T20:52:05.431ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
142422025-11-12T20:52:05.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
142432025-11-12T20:52:05.432ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
142442025-11-12T20:52:05.432ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
142452025-11-12T20:52:05.433ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
142462025-11-12T20:52:05.433ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 154.596941ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
142472025-11-12T20:52:05.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
142482025-11-12T20:52:05.435ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
142492025-11-12T20:52:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
142502025-11-12T20:52:05.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
142512025-11-12T20:52:05.436ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
142522025-11-12T20:52:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
142532025-11-12T20:52:05.438ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
142542025-11-12T20:52:05.438ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
142552025-11-12T20:52:05.439ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "07068735-34c1-42cd-8014-8925e2b1fd35", "content-length": "81", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142562025-11-12T20:52:05.469ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 189.943626ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
142572025-11-12T20:52:05.470ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
142582025-11-12T20:52:05.470ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
142592025-11-12T20:52:05.471ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d068a6a7-2807-4f9f-b984-52c6923718f9", "content-length": "372", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142602025-11-12T20:52:05.471ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142612025-11-12T20:52:05.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142622025-11-12T20:52:05.474ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
142632025-11-12T20:52:05.474ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
142642025-11-12T20:52:05.474ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "36dc217a-eca1-42cb-8607-65923a60c471", "content-length": "26", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142652025-11-12T20:52:05.474ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
142662025-11-12T20:52:05.476ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
142672025-11-12T20:52:05.476ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 200.156029ms iteration = 14 status = {}
142682025-11-12T20:52:05.484ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
142692025-11-12T20:52:05.484ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
142702025-11-12T20:52:05.484ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5422bb4-d6b7-4e86-9560-d51dc28103ba", "content-length": "81", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142712025-11-12T20:52:05.486ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 207.11681ms iteration = 14 status = {"disabled":false,"errors":[],"sps":[]}
142722025-11-12T20:52:05.487ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
142732025-11-12T20:52:05.487ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
142742025-11-12T20:52:05.487ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 207.91821ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
142752025-11-12T20:52:05.494ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
142762025-11-12T20:52:05.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
142772025-11-12T20:52:05.508ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 3629472 } })
142782025-11-12T20:52:05.527ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
142792025-11-12T20:52:05.527ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
142802025-11-12T20:52:05.530ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "49d50be1-7794-4288-941e-9a6ef7306500", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142812025-11-12T20:52:05.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142822025-11-12T20:52:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142832025-11-12T20:52:05.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142842025-11-12T20:52:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142852025-11-12T20:52:05.535ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142862025-11-12T20:52:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142872025-11-12T20:52:05.538ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
142882025-11-12T20:52:05.538ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
142892025-11-12T20:52:05.538ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
142902025-11-12T20:52:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
142912025-11-12T20:52:05.540ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142922025-11-12T20:52:05.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
142932025-11-12T20:52:05.543ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
142942025-11-12T20:52:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
142952025-11-12T20:52:05.544ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142962025-11-12T20:52:05.545ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5bfcc88e-1ffc-47ab-89c2-7f0f769f95fe", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
142972025-11-12T20:52:05.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
142982025-11-12T20:52:05.547ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
142992025-11-12T20:52:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
143002025-11-12T20:52:05.548ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 268.727404ms iteration = 21 status = {}
143012025-11-12T20:52:05.574ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 294.858601ms iteration = 23 status = {"physical_disks_added":0}
143022025-11-12T20:52:05.574ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143032025-11-12T20:52:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143042025-11-12T20:52:05.576ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
143052025-11-12T20:52:05.576ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143062025-11-12T20:52:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143072025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
143082025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
143092025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
143102025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
143112025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143122025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143132025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143142025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
143152025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143162025-11-12T20:52:05.589ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
143172025-11-12T20:52:05.590ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
143182025-11-12T20:52:05.590ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143192025-11-12T20:52:05.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143202025-11-12T20:52:05.592ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
143212025-11-12T20:52:05.592ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
143222025-11-12T20:52:05.658ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
143232025-11-12T20:52:05.658ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca6374fe-6c94-4f26-8473-04c2cd9650b9", "content-length": "691", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
143242025-11-12T20:52:05.658ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
143252025-11-12T20:52:05.658ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
143262025-11-12T20:52:05.658ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
143272025-11-12T20:52:05.660ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 381.059522ms iteration = 14 status = {}
143282025-11-12T20:52:05.706ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4db61e46-7b02-493e-a2fc-5618d8a52aea", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
143292025-11-12T20:52:05.706ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
143302025-11-12T20:52:05.706ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
143312025-11-12T20:52:05.706ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "da5cbce2-7c55-4b3d-9f39-5266d18479f9", "content-length": "682", "date": "Wed, 12 Nov 2025 20:52:05 GMT"} })
143322025-11-12T20:52:05.706ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143332025-11-12T20:52:05.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143342025-11-12T20:52:05.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143352025-11-12T20:52:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143362025-11-12T20:52:05.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143372025-11-12T20:52:05.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143382025-11-12T20:52:05.716ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143392025-11-12T20:52:05.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143402025-11-12T20:52:05.724ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 444.728599ms iteration = 14 status = {}
143412025-11-12T20:52:07.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 420 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 3ef95402-3a17-40b3-90b1-4e2f16595743 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
143422025-11-12T20:52:10.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 80 reason = Timeout
143432025-11-12T20:52:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
143442025-11-12T20:52:10.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
143452025-11-12T20:52:10.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143462025-11-12T20:52:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
143472025-11-12T20:52:10.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
143482025-11-12T20:52:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
143492025-11-12T20:52:10.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
143502025-11-12T20:52:10.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.624397ms iteration = 80 status = {"config_updated":false}
143512025-11-12T20:52:15.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
143522025-11-12T20:52:15.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
143532025-11-12T20:52:15.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
143542025-11-12T20:52:15.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 81 reason = Timeout
143552025-11-12T20:52:15.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
143562025-11-12T20:52:15.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
143572025-11-12T20:52:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
143582025-11-12T20:52:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
143592025-11-12T20:52:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
143602025-11-12T20:52:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143612025-11-12T20:52:15.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143622025-11-12T20:52:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
143632025-11-12T20:52:15.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
143642025-11-12T20:52:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
143652025-11-12T20:52:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
143662025-11-12T20:52:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
143672025-11-12T20:52:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
143682025-11-12T20:52:15.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
143692025-11-12T20:52:15.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.256263ms iteration = 81 status = {"config_updated":false}
143702025-11-12T20:52:15.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
143712025-11-12T20:52:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
143722025-11-12T20:52:15.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143732025-11-12T20:52:15.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
143742025-11-12T20:52:15.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
143752025-11-12T20:52:15.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
143762025-11-12T20:52:15.312ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
143772025-11-12T20:52:15.313ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.212176ms iteration = 41 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
143782025-11-12T20:52:15.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
143792025-11-12T20:52:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
143802025-11-12T20:52:15.326ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 47.217972ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
143812025-11-12T20:52:17.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 373 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = e8b7b1c5-7faa-4493-a130-9fc3d69233ff response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
143822025-11-12T20:52:20.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 36 reason = Timeout
143832025-11-12T20:52:20.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
143842025-11-12T20:52:20.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
143852025-11-12T20:52:20.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 82 reason = Timeout
143862025-11-12T20:52:20.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
143872025-11-12T20:52:20.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
143882025-11-12T20:52:20.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143892025-11-12T20:52:20.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143902025-11-12T20:52:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
143912025-11-12T20:52:20.284ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
143922025-11-12T20:52:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
143932025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
143942025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
143952025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
143962025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb original_time_started = 2025-11-12 20:51:36.705866 UTC
143972025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.508524ms iteration = 36 status = {"Loaded":{"collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","time_loaded":"2025-11-12T20:52:20.278237559Z","time_started":"2025-11-12T20:51:36.705866Z"}}
143982025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 28 reason = Timeout
143992025-11-12T20:52:20.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
144002025-11-12T20:52:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
144012025-11-12T20:52:20.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
144022025-11-12T20:52:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.667509ms iteration = 82 status = {"config_updated":false}
144032025-11-12T20:52:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144042025-11-12T20:52:20.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
144052025-11-12T20:52:20.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
144062025-11-12T20:52:20.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
144072025-11-12T20:52:20.291ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.833131ms iteration = 28 status = "NoSitrep"
144082025-11-12T20:52:20.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 28 reason = Timeout
144092025-11-12T20:52:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
144102025-11-12T20:52:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
144112025-11-12T20:52:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
144122025-11-12T20:52:20.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
144132025-11-12T20:52:20.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144142025-11-12T20:52:20.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
144152025-11-12T20:52:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
144162025-11-12T20:52:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
144172025-11-12T20:52:20.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
144182025-11-12T20:52:20.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
144192025-11-12T20:52:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144202025-11-12T20:52:20.301ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
144212025-11-12T20:52:20.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
144222025-11-12T20:52:20.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.162103ms iteration = 28 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
144232025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
144242025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
144252025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144262025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
144272025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 83 reason = Timeout
144282025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
144292025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
144302025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
144312025-11-12T20:52:25.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
144322025-11-12T20:52:25.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
144332025-11-12T20:52:25.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144342025-11-12T20:52:25.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
144352025-11-12T20:52:25.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144362025-11-12T20:52:25.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.214343ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
144372025-11-12T20:52:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
144382025-11-12T20:52:25.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
144392025-11-12T20:52:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
144402025-11-12T20:52:25.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
144412025-11-12T20:52:25.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.435922ms iteration = 83 status = {"config_updated":false}
144422025-11-12T20:52:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
144432025-11-12T20:52:25.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144442025-11-12T20:52:25.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
144452025-11-12T20:52:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144462025-11-12T20:52:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
144472025-11-12T20:52:25.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144482025-11-12T20:52:25.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
144492025-11-12T20:52:25.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
144502025-11-12T20:52:25.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
144512025-11-12T20:52:25.309ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
144522025-11-12T20:52:25.309ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.187231ms iteration = 42 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
144532025-11-12T20:52:27.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 276 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = d6303984-46d7-452d-b865-1a8b07a6d3cb response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
144542025-11-12T20:52:30.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 84 reason = Timeout
144552025-11-12T20:52:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
144562025-11-12T20:52:30.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
144572025-11-12T20:52:30.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144582025-11-12T20:52:30.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
144592025-11-12T20:52:30.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
144602025-11-12T20:52:30.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
144612025-11-12T20:52:30.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
144622025-11-12T20:52:30.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.437387ms iteration = 84 status = {"config_updated":false}
144632025-11-12T20:52:32.915ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:51395
144642025-11-12T20:52:32.915ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = af292662-5921-4d1f-ab7f-f74764381b01 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
144652025-11-12T20:52:32.915ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = af292662-5921-4d1f-ab7f-f74764381b01 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
144662025-11-12T20:52:32.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = af292662-5921-4d1f-ab7f-f74764381b01 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
144672025-11-12T20:52:32.918ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2816 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = af292662-5921-4d1f-ab7f-f74764381b01 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
144682025-11-12T20:52:32.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = e4b5f85a-78d2-4640-a6d9-5bf64c780435 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
144692025-11-12T20:52:32.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = e4b5f85a-78d2-4640-a6d9-5bf64c780435 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
144702025-11-12T20:52:32.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = e4b5f85a-78d2-4640-a6d9-5bf64c780435 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
144712025-11-12T20:52:32.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2829 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = e4b5f85a-78d2-4640-a6d9-5bf64c780435 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
144722025-11-12T20:52:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
144732025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
144742025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
144752025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
144762025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
144772025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
144782025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
144792025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
144802025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
144812025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
144822025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
144832025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
144842025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 15 reason = Timeout
144852025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
144862025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
144872025-11-12T20:52:35.276ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:42:35.276072059 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
144882025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
144892025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
144902025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
144912025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
144922025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
144932025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
144942025-11-12T20:52:35.276ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
144952025-11-12T20:52:35.276ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = ExternalNetworking step id = Ensure
144962025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144972025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
144982025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
144992025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
145002025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
145012025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
145022025-11-12T20:52:35.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1350, tv_nsec: 772285352 } })
145032025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145042025-11-12T20:52:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
145052025-11-12T20:52:35.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1350, tv_nsec: 772388845 } })
145062025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
145072025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.428457ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
145082025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
145092025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.513789ms iteration = 8 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
145102025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 37 reason = Timeout
145112025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
145122025-11-12T20:52:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
145132025-11-12T20:52:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
145142025-11-12T20:52:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
145152025-11-12T20:52:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
145162025-11-12T20:52:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
145172025-11-12T20:52:35.278ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145182025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
145192025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
145202025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
145212025-11-12T20:52:35.279ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
145222025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
145232025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
145242025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
145252025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 15 reason = Timeout
145262025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
145272025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
145282025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
145292025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
145302025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
145312025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
145322025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
145332025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
145342025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
145352025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
145362025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
145372025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
145382025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
145392025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
145402025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
145412025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 19 reason = Timeout
145422025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
145432025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
145442025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
145452025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
145462025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
145472025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
145482025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
145492025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
145502025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
145512025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
145522025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
145532025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
145542025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
145552025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
145562025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
145572025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
145582025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
145592025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
145602025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
145612025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
145622025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
145632025-11-12T20:52:35.279ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
145642025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
145652025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
145662025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
145672025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
145682025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
145692025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 85 reason = Timeout
145702025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
145712025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
145722025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
145732025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
145742025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
145752025-11-12T20:52:35.279ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:42:35.276072059 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
145762025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
145772025-11-12T20:52:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
145782025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.902271ms iteration = 8 status = {"expiration":"2025-11-12T20:42:35.276072059Z","pruned":[]}
145792025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
145802025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
145812025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
145822025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
145832025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
145842025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
145852025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
145862025-11-12T20:52:35.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1350, tv_nsec: 775794703 } })
145872025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
145882025-11-12T20:52:35.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1350, tv_nsec: 775819013 } })
145892025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
145902025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
145912025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
145922025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
145932025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
145942025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
145952025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
145962025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
145972025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
145982025-11-12T20:52:35.280ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1350, tv_nsec: 776173383 } })
145992025-11-12T20:52:35.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146002025-11-12T20:52:35.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
146012025-11-12T20:52:35.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146022025-11-12T20:52:35.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146032025-11-12T20:52:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
146042025-11-12T20:52:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
146052025-11-12T20:52:35.282ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146062025-11-12T20:52:35.282ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.711094ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
146072025-11-12T20:52:35.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146082025-11-12T20:52:35.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.266374ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
146092025-11-12T20:52:35.283ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
146102025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
146112025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
146122025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
146132025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
146142025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
146152025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
146162025-11-12T20:52:35.284ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
146172025-11-12T20:52:35.284ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.235395ms iteration = 8 status = {"by_rx":{},"error":null}
146182025-11-12T20:52:35.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146192025-11-12T20:52:35.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146202025-11-12T20:52:35.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 29 reason = Timeout
146212025-11-12T20:52:35.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
146222025-11-12T20:52:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
146232025-11-12T20:52:35.287ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.544275ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
146242025-11-12T20:52:35.287ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146252025-11-12T20:52:35.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146262025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
146272025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
146282025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146292025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
146302025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
146312025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
146322025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
146332025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
146342025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
146352025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
146362025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
146372025-11-12T20:52:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
146382025-11-12T20:52:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
146392025-11-12T20:52:35.290ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 10.274196ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
146402025-11-12T20:52:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
146412025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146422025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 29 reason = Timeout
146432025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
146442025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 8 reason = Timeout
146452025-11-12T20:52:35.292ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
146462025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
146472025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
146482025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
146492025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
146502025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
146512025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
146522025-11-12T20:52:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
146532025-11-12T20:52:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
146542025-11-12T20:52:35.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
146552025-11-12T20:52:35.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146562025-11-12T20:52:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
146572025-11-12T20:52:35.293ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
146582025-11-12T20:52:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
146592025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
146602025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
146612025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
146622025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
146632025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
146642025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
146652025-11-12T20:52:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
146662025-11-12T20:52:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
146672025-11-12T20:52:35.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
146682025-11-12T20:52:35.295ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
146692025-11-12T20:52:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
146702025-11-12T20:52:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
146712025-11-12T20:52:35.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
146722025-11-12T20:52:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
146732025-11-12T20:52:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
146742025-11-12T20:52:35.297ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
146752025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
146762025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146772025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
146782025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
146792025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
146802025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
146812025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
146822025-11-12T20:52:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
146832025-11-12T20:52:35.299ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146842025-11-12T20:52:35.300ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb original_time_started = 2025-11-12 20:51:36.705866 UTC
146852025-11-12T20:52:35.300ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 22.589465ms iteration = 37 status = {"Loaded":{"collection_id":"5b8b6c4c-fea2-4427-b744-2d0ccac60ceb","time_loaded":"2025-11-12T20:52:35.277797834Z","time_started":"2025-11-12T20:51:36.705866Z"}}
146862025-11-12T20:52:35.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146872025-11-12T20:52:35.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
146882025-11-12T20:52:35.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
146892025-11-12T20:52:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
146902025-11-12T20:52:35.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
146912025-11-12T20:52:35.304ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
146922025-11-12T20:52:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
146932025-11-12T20:52:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146942025-11-12T20:52:35.308ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
146952025-11-12T20:52:35.308ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 28.528178ms iteration = 85 status = {"config_updated":false}
146962025-11-12T20:52:35.318ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
146972025-11-12T20:52:35.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146982025-11-12T20:52:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
146992025-11-12T20:52:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
147002025-11-12T20:52:35.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147012025-11-12T20:52:35.323ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1350, tv_nsec: 818691173 } })
147022025-11-12T20:52:35.323ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
147032025-11-12T20:52:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
147042025-11-12T20:52:35.324ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
147052025-11-12T20:52:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
147062025-11-12T20:52:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
147072025-11-12T20:52:35.326ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
147082025-11-12T20:52:35.326ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 46.413891ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
147092025-11-12T20:52:35.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147102025-11-12T20:52:35.328ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
147112025-11-12T20:52:35.329ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 49.753678ms iteration = 15 status = 0
147122025-11-12T20:52:35.330ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 43.770733ms iteration = 29 status = "NoSitrep"
147132025-11-12T20:52:35.331ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
147142025-11-12T20:52:35.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
147152025-11-12T20:52:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
147162025-11-12T20:52:35.331ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
147172025-11-12T20:52:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
147182025-11-12T20:52:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
147192025-11-12T20:52:35.332ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147202025-11-12T20:52:35.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147212025-11-12T20:52:35.334ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
147222025-11-12T20:52:35.334ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
147232025-11-12T20:52:35.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
147242025-11-12T20:52:35.336ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
147252025-11-12T20:52:35.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
147262025-11-12T20:52:35.337ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
147272025-11-12T20:52:35.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
147282025-11-12T20:52:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
147292025-11-12T20:52:35.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147302025-11-12T20:52:35.339ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
147312025-11-12T20:52:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
147322025-11-12T20:52:35.339ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
147332025-11-12T20:52:35.339ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
147342025-11-12T20:52:35.339ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
147352025-11-12T20:52:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
147362025-11-12T20:52:35.341ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
147372025-11-12T20:52:35.342ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147382025-11-12T20:52:35.343ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
147392025-11-12T20:52:35.345ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
147402025-11-12T20:52:35.346ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
147412025-11-12T20:52:35.346ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 53.859413ms iteration = 29 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
147422025-11-12T20:52:35.346ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
147432025-11-12T20:52:35.349ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
147442025-11-12T20:52:35.349ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
147452025-11-12T20:52:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
147462025-11-12T20:52:35.351ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147472025-11-12T20:52:35.353ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147482025-11-12T20:52:35.354ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
147492025-11-12T20:52:35.355ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = SledList step id = Fetch
147502025-11-12T20:52:35.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
147512025-11-12T20:52:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
147522025-11-12T20:52:35.357ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
147532025-11-12T20:52:35.357ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
147542025-11-12T20:52:35.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
147552025-11-12T20:52:35.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
147562025-11-12T20:52:35.358ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
147572025-11-12T20:52:35.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
147582025-11-12T20:52:35.359ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147592025-11-12T20:52:35.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147602025-11-12T20:52:35.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
147612025-11-12T20:52:35.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
147622025-11-12T20:52:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
147632025-11-12T20:52:35.364ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147642025-11-12T20:52:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
147652025-11-12T20:52:35.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147662025-11-12T20:52:35.366ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
147672025-11-12T20:52:35.366ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
147682025-11-12T20:52:35.366ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 86.343434ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
147692025-11-12T20:52:35.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147702025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
147712025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
147722025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
147732025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
147742025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
147752025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
147762025-11-12T20:52:35.370ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 94.629289ms iteration = 8 status = {"generation":1}
147772025-11-12T20:52:35.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147782025-11-12T20:52:35.377ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
147792025-11-12T20:52:35.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
147802025-11-12T20:52:35.378ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
147812025-11-12T20:52:35.378ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147822025-11-12T20:52:35.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
147832025-11-12T20:52:35.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147842025-11-12T20:52:35.381ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147852025-11-12T20:52:35.382ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 103.335464ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
147862025-11-12T20:52:35.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147872025-11-12T20:52:35.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147882025-11-12T20:52:35.383ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = DeployNexusRecords step id = Ensure
147892025-11-12T20:52:35.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
147902025-11-12T20:52:35.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147912025-11-12T20:52:35.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
147922025-11-12T20:52:35.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147932025-11-12T20:52:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
147942025-11-12T20:52:35.387ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147952025-11-12T20:52:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147962025-11-12T20:52:35.388ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
147972025-11-12T20:52:35.388ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
147982025-11-12T20:52:35.388ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
147992025-11-12T20:52:35.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
148002025-11-12T20:52:35.390ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
148012025-11-12T20:52:35.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
148022025-11-12T20:52:35.390ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 111.122336ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
148032025-11-12T20:52:35.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
148042025-11-12T20:52:35.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148052025-11-12T20:52:35.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148062025-11-12T20:52:35.393ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148072025-11-12T20:52:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
148082025-11-12T20:52:35.397ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
148092025-11-12T20:52:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
148102025-11-12T20:52:35.404ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
148112025-11-12T20:52:35.404ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
148122025-11-12T20:52:35.404ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
148132025-11-12T20:52:35.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
148142025-11-12T20:52:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
148152025-11-12T20:52:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148162025-11-12T20:52:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148172025-11-12T20:52:35.407ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 127.880519ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
148182025-11-12T20:52:35.408ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
148192025-11-12T20:52:35.408ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
148202025-11-12T20:52:35.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148212025-11-12T20:52:35.408ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 128.560127ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
148222025-11-12T20:52:35.408ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
148232025-11-12T20:52:35.408ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 129.346847ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
148242025-11-12T20:52:35.410ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148252025-11-12T20:52:35.411ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
148262025-11-12T20:52:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 131.559834ms iteration = 43 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
148272025-11-12T20:52:35.412ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
148282025-11-12T20:52:35.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148292025-11-12T20:52:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148302025-11-12T20:52:35.415ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
148312025-11-12T20:52:35.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148322025-11-12T20:52:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
148332025-11-12T20:52:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
148342025-11-12T20:52:35.418ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
148352025-11-12T20:52:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
148362025-11-12T20:52:35.419ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 140.104815ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
148372025-11-12T20:52:35.420ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
148382025-11-12T20:52:35.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148392025-11-12T20:52:35.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148402025-11-12T20:52:35.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148412025-11-12T20:52:35.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148422025-11-12T20:52:35.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148432025-11-12T20:52:35.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148442025-11-12T20:52:35.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148452025-11-12T20:52:35.434ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
148462025-11-12T20:52:35.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
148472025-11-12T20:52:35.437ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148482025-11-12T20:52:35.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
148492025-11-12T20:52:35.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148502025-11-12T20:52:35.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148512025-11-12T20:52:35.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
148522025-11-12T20:52:35.442ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
148532025-11-12T20:52:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
148542025-11-12T20:52:35.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148552025-11-12T20:52:35.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148562025-11-12T20:52:35.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148572025-11-12T20:52:35.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
148582025-11-12T20:52:35.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148592025-11-12T20:52:35.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148602025-11-12T20:52:35.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148612025-11-12T20:52:35.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
148622025-11-12T20:52:35.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148632025-11-12T20:52:35.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148642025-11-12T20:52:35.466ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
148652025-11-12T20:52:35.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
148662025-11-12T20:52:35.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148672025-11-12T20:52:35.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148682025-11-12T20:52:35.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148692025-11-12T20:52:35.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148702025-11-12T20:52:35.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
148712025-11-12T20:52:35.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148722025-11-12T20:52:35.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148732025-11-12T20:52:35.485ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
148742025-11-12T20:52:35.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
148752025-11-12T20:52:35.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
148762025-11-12T20:52:35.487ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
148772025-11-12T20:52:35.487ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
148782025-11-12T20:52:35.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
148792025-11-12T20:52:35.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
148802025-11-12T20:52:35.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
148812025-11-12T20:52:35.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148822025-11-12T20:52:35.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
148832025-11-12T20:52:35.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148842025-11-12T20:52:35.489ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
148852025-11-12T20:52:35.489ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 209.755229ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
148862025-11-12T20:52:35.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148872025-11-12T20:52:35.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148882025-11-12T20:52:35.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
148892025-11-12T20:52:35.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148902025-11-12T20:52:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148912025-11-12T20:52:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
148922025-11-12T20:52:35.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148932025-11-12T20:52:35.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148942025-11-12T20:52:35.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148952025-11-12T20:52:35.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
148962025-11-12T20:52:35.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148972025-11-12T20:52:35.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148982025-11-12T20:52:35.511ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
148992025-11-12T20:52:35.511ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
149002025-11-12T20:52:35.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
149012025-11-12T20:52:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
149022025-11-12T20:52:35.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149032025-11-12T20:52:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
149042025-11-12T20:52:35.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
149052025-11-12T20:52:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
149062025-11-12T20:52:35.524ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
149072025-11-12T20:52:35.524ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
149082025-11-12T20:52:35.524ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
149092025-11-12T20:52:35.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 248.963793ms iteration = 8 status = {"generation":2}
149102025-11-12T20:52:35.539ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
149112025-11-12T20:52:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
149122025-11-12T20:52:35.542ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
149132025-11-12T20:52:35.542ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 266.986829ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
149142025-11-12T20:52:35.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149152025-11-12T20:52:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149162025-11-12T20:52:35.553ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
149172025-11-12T20:52:35.553ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
149182025-11-12T20:52:35.553ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 274.501134ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
149192025-11-12T20:52:35.554ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 275.312595ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
149202025-11-12T20:52:35.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149212025-11-12T20:52:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149222025-11-12T20:52:35.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149232025-11-12T20:52:35.565ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
149242025-11-12T20:52:35.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149252025-11-12T20:52:35.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149262025-11-12T20:52:35.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149272025-11-12T20:52:35.572ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
149282025-11-12T20:52:35.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149292025-11-12T20:52:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149302025-11-12T20:52:35.573ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149312025-11-12T20:52:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
149322025-11-12T20:52:35.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149332025-11-12T20:52:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149342025-11-12T20:52:35.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149352025-11-12T20:52:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149362025-11-12T20:52:35.586ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
149372025-11-12T20:52:35.586ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149382025-11-12T20:52:35.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149392025-11-12T20:52:35.586ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "18bd9ee1-b8d0-45c7-8400-3c088c81c923", "content-length": "26", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
149402025-11-12T20:52:35.586ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149412025-11-12T20:52:35.587ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
149422025-11-12T20:52:35.587ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
149432025-11-12T20:52:35.587ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149442025-11-12T20:52:35.588ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b1f1aa0-c235-4153-b163-04356fc014ab", "content-length": "26", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
149452025-11-12T20:52:35.588ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149462025-11-12T20:52:35.589ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = SledAgent step id = Ensure
149472025-11-12T20:52:35.589ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
149482025-11-12T20:52:35.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
149492025-11-12T20:52:35.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149502025-11-12T20:52:35.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149512025-11-12T20:52:35.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149522025-11-12T20:52:35.594ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
149532025-11-12T20:52:35.597ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
149542025-11-12T20:52:35.597ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 321.564332ms iteration = 15 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
149552025-11-12T20:52:35.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
149562025-11-12T20:52:35.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149572025-11-12T20:52:35.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149582025-11-12T20:52:35.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149592025-11-12T20:52:35.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149602025-11-12T20:52:35.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149612025-11-12T20:52:35.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149622025-11-12T20:52:35.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149632025-11-12T20:52:35.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
149642025-11-12T20:52:35.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149652025-11-12T20:52:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149662025-11-12T20:52:35.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149672025-11-12T20:52:35.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149682025-11-12T20:52:35.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149692025-11-12T20:52:35.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
149702025-11-12T20:52:35.623ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 343.961112ms iteration = 24 status = {"physical_disks_added":0}
149712025-11-12T20:52:35.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149722025-11-12T20:52:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149732025-11-12T20:52:35.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149742025-11-12T20:52:35.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149752025-11-12T20:52:35.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149762025-11-12T20:52:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
149772025-11-12T20:52:35.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149782025-11-12T20:52:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149792025-11-12T20:52:35.638ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
149802025-11-12T20:52:35.638ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
149812025-11-12T20:52:35.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149822025-11-12T20:52:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149832025-11-12T20:52:35.639ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e318c24-2521-4701-9602-4387cdf1d1aa", "content-length": "372", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
149842025-11-12T20:52:35.639ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
149852025-11-12T20:52:35.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
149862025-11-12T20:52:35.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149872025-11-12T20:52:35.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
149882025-11-12T20:52:35.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149892025-11-12T20:52:35.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149902025-11-12T20:52:35.651ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
149912025-11-12T20:52:35.653ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "1f1da607-a2ee-44d8-93e4-fa838c860304", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
149922025-11-12T20:52:35.654ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
149932025-11-12T20:52:35.656ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
149942025-11-12T20:52:35.656ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 364.044471ms iteration = 8 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
149952025-11-12T20:52:35.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
149962025-11-12T20:52:35.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
149972025-11-12T20:52:35.659ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149982025-11-12T20:52:35.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149992025-11-12T20:52:35.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150002025-11-12T20:52:35.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150012025-11-12T20:52:35.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
150022025-11-12T20:52:35.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
150032025-11-12T20:52:35.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
150042025-11-12T20:52:35.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
150052025-11-12T20:52:35.666ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150062025-11-12T20:52:35.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150072025-11-12T20:52:35.666ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
150082025-11-12T20:52:35.666ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
150092025-11-12T20:52:35.667ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "376ef520-3eff-490e-bcdc-dfe9e147a36c", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
150102025-11-12T20:52:35.667ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150112025-11-12T20:52:35.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150122025-11-12T20:52:35.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
150132025-11-12T20:52:35.668ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150142025-11-12T20:52:35.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
150152025-11-12T20:52:35.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150162025-11-12T20:52:35.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150172025-11-12T20:52:35.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150182025-11-12T20:52:35.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
150192025-11-12T20:52:35.671ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150202025-11-12T20:52:35.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150212025-11-12T20:52:35.673ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150222025-11-12T20:52:35.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150232025-11-12T20:52:35.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
150242025-11-12T20:52:35.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
150252025-11-12T20:52:35.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
150262025-11-12T20:52:35.677ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
150272025-11-12T20:52:35.677ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
150282025-11-12T20:52:35.677ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 1f55947c-5394-4ed5-94e5-d44f567b5b23 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
150292025-11-12T20:52:35.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
150302025-11-12T20:52:35.677ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
150312025-11-12T20:52:35.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
150322025-11-12T20:52:35.677ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
150332025-11-12T20:52:35.678ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6d3c40e8-8ff2-488e-afa1-b1f2fc726d8b", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
150342025-11-12T20:52:35.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150352025-11-12T20:52:35.681ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 402.221239ms iteration = 22 status = {}
150362025-11-12T20:52:35.681ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Dependency
150372025-11-12T20:52:35.681ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
150382025-11-12T20:52:35.681ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150392025-11-12T20:52:35.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150402025-11-12T20:52:35.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150412025-11-12T20:52:35.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
150422025-11-12T20:52:35.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
150432025-11-12T20:52:35.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
150442025-11-12T20:52:35.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
150452025-11-12T20:52:35.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150462025-11-12T20:52:35.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150472025-11-12T20:52:35.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150482025-11-12T20:52:35.689ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
150492025-11-12T20:52:35.689ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
150502025-11-12T20:52:35.689ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 410.126334ms iteration = 19 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
150512025-11-12T20:52:35.691ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
150522025-11-12T20:52:35.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
150532025-11-12T20:52:35.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150542025-11-12T20:52:35.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150552025-11-12T20:52:35.739ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
150562025-11-12T20:52:35.740ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1fe2913-8c17-4a15-a978-e563dad0c466", "content-length": "78", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
150572025-11-12T20:52:35.740ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
150582025-11-12T20:52:35.740ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
150592025-11-12T20:52:35.740ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
150602025-11-12T20:52:35.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
150612025-11-12T20:52:35.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
150622025-11-12T20:52:35.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
150632025-11-12T20:52:35.743ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
150642025-11-12T20:52:35.743ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
150652025-11-12T20:52:35.747ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
150662025-11-12T20:52:35.747ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
150672025-11-12T20:52:35.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
150682025-11-12T20:52:35.749ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
150692025-11-12T20:52:35.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
150702025-11-12T20:52:35.749ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
150712025-11-12T20:52:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
150722025-11-12T20:52:35.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
150732025-11-12T20:52:35.753ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
150742025-11-12T20:52:35.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
150752025-11-12T20:52:35.756ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
150762025-11-12T20:52:35.756ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
150772025-11-12T20:52:35.757ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "795cf709-00a9-4785-bba9-efe233fcd6a2", "content-length": "81", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
150782025-11-12T20:52:35.782ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
150792025-11-12T20:52:35.782ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 503.246163ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
150802025-11-12T20:52:35.801ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
150812025-11-12T20:52:35.801ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
150822025-11-12T20:52:35.802ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3dbc351-d9cb-4949-834c-f3489d00153d", "content-length": "81", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
150832025-11-12T20:52:35.807ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
150842025-11-12T20:52:35.807ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
150852025-11-12T20:52:35.807ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9fe0c6e-9406-4f0f-b934-9ebacce6ff0a", "content-length": "81", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
150862025-11-12T20:52:35.809ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 529.76466ms iteration = 15 status = {"disabled":false,"errors":[],"sps":[]}
150872025-11-12T20:52:35.813ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
150882025-11-12T20:52:35.815ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "15b04cda-cad9-4764-bf2c-4c956fd7b947", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
150892025-11-12T20:52:35.817ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = FirewallRules step id = Ensure
150902025-11-12T20:52:35.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
150912025-11-12T20:52:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
150922025-11-12T20:52:35.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
150932025-11-12T20:52:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
150942025-11-12T20:52:35.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
150952025-11-12T20:52:35.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
150962025-11-12T20:52:35.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
150972025-11-12T20:52:35.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
150982025-11-12T20:52:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
150992025-11-12T20:52:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151002025-11-12T20:52:35.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151012025-11-12T20:52:35.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151022025-11-12T20:52:35.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151032025-11-12T20:52:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
151042025-11-12T20:52:35.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151052025-11-12T20:52:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151062025-11-12T20:52:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151072025-11-12T20:52:35.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151082025-11-12T20:52:35.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151092025-11-12T20:52:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151102025-11-12T20:52:35.837ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
151112025-11-12T20:52:35.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151122025-11-12T20:52:35.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
151132025-11-12T20:52:35.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151142025-11-12T20:52:35.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151152025-11-12T20:52:35.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151162025-11-12T20:52:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151172025-11-12T20:52:35.846ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
151182025-11-12T20:52:35.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151192025-11-12T20:52:35.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151202025-11-12T20:52:35.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151212025-11-12T20:52:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151222025-11-12T20:52:35.850ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 345607899 } })
151232025-11-12T20:52:35.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151242025-11-12T20:52:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151252025-11-12T20:52:35.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151262025-11-12T20:52:35.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151272025-11-12T20:52:35.853ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c698ddae-0b38-4500-8856-de11382d198a", "content-length": "691", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
151282025-11-12T20:52:35.853ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
151292025-11-12T20:52:35.853ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
151302025-11-12T20:52:35.853ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
151312025-11-12T20:52:35.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151322025-11-12T20:52:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151332025-11-12T20:52:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151342025-11-12T20:52:35.857ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 578.08361ms iteration = 15 status = {}
151352025-11-12T20:52:35.858ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 582.194956ms iteration = 15 status = {}
151362025-11-12T20:52:35.861ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
151372025-11-12T20:52:35.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151382025-11-12T20:52:35.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151392025-11-12T20:52:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151402025-11-12T20:52:35.863ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
151412025-11-12T20:52:35.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151422025-11-12T20:52:35.863ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "97cd0b2c-062c-4a6a-99c3-e4bfae10c15c", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
151432025-11-12T20:52:35.863ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
151442025-11-12T20:52:35.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151452025-11-12T20:52:35.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151462025-11-12T20:52:35.865ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c3f31324-f713-4c66-9dde-3d132d201dd8", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
151472025-11-12T20:52:35.865ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
151482025-11-12T20:52:35.867ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
151492025-11-12T20:52:35.867ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 591.871347ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
151502025-11-12T20:52:35.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151512025-11-12T20:52:35.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151522025-11-12T20:52:35.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151532025-11-12T20:52:35.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151542025-11-12T20:52:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151552025-11-12T20:52:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151562025-11-12T20:52:35.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151572025-11-12T20:52:35.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151582025-11-12T20:52:35.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151592025-11-12T20:52:35.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
151602025-11-12T20:52:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151612025-11-12T20:52:35.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151622025-11-12T20:52:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151632025-11-12T20:52:35.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151642025-11-12T20:52:35.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151652025-11-12T20:52:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151662025-11-12T20:52:35.887ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
151672025-11-12T20:52:35.887ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
151682025-11-12T20:52:35.887ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5533982-bf10-46db-aa74-c43e55f932c8", "content-length": "372", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
151692025-11-12T20:52:35.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151702025-11-12T20:52:35.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151712025-11-12T20:52:35.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151722025-11-12T20:52:35.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
151732025-11-12T20:52:35.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151742025-11-12T20:52:35.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151752025-11-12T20:52:35.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151762025-11-12T20:52:35.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151772025-11-12T20:52:35.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151782025-11-12T20:52:35.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151792025-11-12T20:52:35.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151802025-11-12T20:52:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151812025-11-12T20:52:35.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151822025-11-12T20:52:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151832025-11-12T20:52:35.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151842025-11-12T20:52:35.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151852025-11-12T20:52:35.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151862025-11-12T20:52:35.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151872025-11-12T20:52:35.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151882025-11-12T20:52:35.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151892025-11-12T20:52:35.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151902025-11-12T20:52:35.916ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
151912025-11-12T20:52:35.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151922025-11-12T20:52:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151932025-11-12T20:52:35.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151942025-11-12T20:52:35.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151952025-11-12T20:52:35.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
151962025-11-12T20:52:35.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
151972025-11-12T20:52:35.920ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
151982025-11-12T20:52:35.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151992025-11-12T20:52:35.921ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "10b355b0-1c15-4a19-b6be-9250f023ad56", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152002025-11-12T20:52:35.921ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152012025-11-12T20:52:35.923ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "52819bcc-12ea-4e2e-a7fc-ec263434c77a", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152022025-11-12T20:52:35.924ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152032025-11-12T20:52:35.925ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3ada92bc-d3c2-4c55-99cd-e61ae3131c20", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152042025-11-12T20:52:35.925ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152052025-11-12T20:52:35.926ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
152062025-11-12T20:52:35.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152072025-11-12T20:52:35.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152082025-11-12T20:52:35.928ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
152092025-11-12T20:52:35.928ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 652.214879ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
152102025-11-12T20:52:35.932ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
152112025-11-12T20:52:35.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152122025-11-12T20:52:35.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152132025-11-12T20:52:35.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152142025-11-12T20:52:35.936ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
152152025-11-12T20:52:35.936ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
152162025-11-12T20:52:35.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
152172025-11-12T20:52:35.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152182025-11-12T20:52:35.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152192025-11-12T20:52:35.938ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
152202025-11-12T20:52:35.938ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
152212025-11-12T20:52:35.938ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
152222025-11-12T20:52:35.941ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "866a8e53-b008-4508-841e-c59a6e1fd57e", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
152232025-11-12T20:52:35.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152242025-11-12T20:52:35.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152252025-11-12T20:52:35.943ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152262025-11-12T20:52:35.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152272025-11-12T20:52:35.946ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152282025-11-12T20:52:35.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152292025-11-12T20:52:35.948ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
152302025-11-12T20:52:35.949ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
152312025-11-12T20:52:35.949ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7198b35b-6cb3-4538-992b-4b7e85dd7a59", "date": "Wed, 12 Nov 2025 20:52:35 GMT"} })
152322025-11-12T20:52:35.951ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 269.904194ms iteration = 23 status = {}
152332025-11-12T20:52:35.955ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
152342025-11-12T20:52:35.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
152352025-11-12T20:52:35.956ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
152362025-11-12T20:52:35.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
152372025-11-12T20:52:35.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
152382025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
152392025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
152402025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
152412025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
152422025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152432025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152442025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152452025-11-12T20:52:35.975ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
152462025-11-12T20:52:35.976ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152472025-11-12T20:52:35.976ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
152482025-11-12T20:52:35.976ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
152492025-11-12T20:52:35.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
152502025-11-12T20:52:35.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
152512025-11-12T20:52:35.978ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
152522025-11-12T20:52:35.978ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
152532025-11-12T20:52:36.003ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
152542025-11-12T20:52:36.003ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
152552025-11-12T20:52:36.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152562025-11-12T20:52:36.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152572025-11-12T20:52:36.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152582025-11-12T20:52:36.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152592025-11-12T20:52:36.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152602025-11-12T20:52:36.044ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
152612025-11-12T20:52:36.044ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
152622025-11-12T20:52:36.045ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3ab17e6d-c64c-42fc-8a53-ac651055d58b", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
152632025-11-12T20:52:36.047ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
152642025-11-12T20:52:36.047ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = Dns step id = Ensure
152652025-11-12T20:52:36.047ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
152662025-11-12T20:52:36.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152672025-11-12T20:52:36.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152682025-11-12T20:52:36.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152692025-11-12T20:52:36.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152702025-11-12T20:52:36.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152712025-11-12T20:52:36.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
152722025-11-12T20:52:36.053ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
152732025-11-12T20:52:36.053ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
152742025-11-12T20:52:36.053ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
152752025-11-12T20:52:36.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152762025-11-12T20:52:36.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152772025-11-12T20:52:36.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152782025-11-12T20:52:36.058ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
152792025-11-12T20:52:36.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152802025-11-12T20:52:36.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152812025-11-12T20:52:36.061ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
152822025-11-12T20:52:36.061ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
152832025-11-12T20:52:36.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152842025-11-12T20:52:36.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152852025-11-12T20:52:36.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152862025-11-12T20:52:36.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
152872025-11-12T20:52:36.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152882025-11-12T20:52:36.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152892025-11-12T20:52:36.068ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
152902025-11-12T20:52:36.068ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
152912025-11-12T20:52:36.068ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
152922025-11-12T20:52:36.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152932025-11-12T20:52:36.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152942025-11-12T20:52:36.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
152952025-11-12T20:52:36.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
152962025-11-12T20:52:36.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152972025-11-12T20:52:36.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
152982025-11-12T20:52:36.093ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1c9e95ca-4783-46ca-9dc8-bb842eacff75", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
152992025-11-12T20:52:36.093ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
153002025-11-12T20:52:36.093ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
153012025-11-12T20:52:36.099ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc2d6387-46c3-42b5-ae62-4193b73d74f5", "content-length": "682", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
153022025-11-12T20:52:36.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153032025-11-12T20:52:36.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153042025-11-12T20:52:36.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153052025-11-12T20:52:36.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153062025-11-12T20:52:36.108ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153072025-11-12T20:52:36.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153082025-11-12T20:52:36.110ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153092025-11-12T20:52:36.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153102025-11-12T20:52:36.112ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
153112025-11-12T20:52:36.112ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
153122025-11-12T20:52:36.112ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
153132025-11-12T20:52:36.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153142025-11-12T20:52:36.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153152025-11-12T20:52:36.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153162025-11-12T20:52:36.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
153172025-11-12T20:52:36.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153182025-11-12T20:52:36.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153192025-11-12T20:52:36.118ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 839.213288ms iteration = 15 status = {}
153202025-11-12T20:52:36.120ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
153212025-11-12T20:52:36.120ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
153222025-11-12T20:52:36.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153232025-11-12T20:52:36.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153242025-11-12T20:52:36.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153252025-11-12T20:52:36.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
153262025-11-12T20:52:36.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153272025-11-12T20:52:36.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153282025-11-12T20:52:36.127ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
153292025-11-12T20:52:36.127ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
153302025-11-12T20:52:36.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153312025-11-12T20:52:36.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153322025-11-12T20:52:36.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153332025-11-12T20:52:36.131ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
153342025-11-12T20:52:36.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153352025-11-12T20:52:36.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153362025-11-12T20:52:36.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153372025-11-12T20:52:36.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153382025-11-12T20:52:36.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153392025-11-12T20:52:36.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153402025-11-12T20:52:36.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153412025-11-12T20:52:36.139ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
153422025-11-12T20:52:36.141ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
153432025-11-12T20:52:36.141ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
153442025-11-12T20:52:36.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = OmicronZones step id = Cleanup
153452025-11-12T20:52:36.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = OmicronZones step id = Cleanup
153462025-11-12T20:52:36.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = PhysicalDisks step id = Cleanup
153472025-11-12T20:52:36.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = Clickhouse step id = Ensure
153482025-11-12T20:52:36.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = Clickhouse step id = Ensure
153492025-11-12T20:52:36.173ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
153502025-11-12T20:52:36.178ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "601ba595-b752-4fbc-a905-a6b53307489b", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
153512025-11-12T20:52:36.179ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = SupportBundles step id = Cleanup
153522025-11-12T20:52:36.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153532025-11-12T20:52:36.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153542025-11-12T20:52:36.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153552025-11-12T20:52:36.182ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
153562025-11-12T20:52:36.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153572025-11-12T20:52:36.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153582025-11-12T20:52:36.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153592025-11-12T20:52:36.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153602025-11-12T20:52:36.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153612025-11-12T20:52:36.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
153622025-11-12T20:52:36.193ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = OmicronZones step id = Cleanup
153632025-11-12T20:52:36.193ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
153642025-11-12T20:52:36.193ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
153652025-11-12T20:52:36.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153662025-11-12T20:52:36.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153672025-11-12T20:52:36.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153682025-11-12T20:52:36.196ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
153692025-11-12T20:52:36.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153702025-11-12T20:52:36.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153712025-11-12T20:52:36.235ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
153722025-11-12T20:52:36.235ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
153732025-11-12T20:52:36.235ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = Cockroach step id = Ensure
153742025-11-12T20:52:36.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153752025-11-12T20:52:36.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153762025-11-12T20:52:36.250ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
153772025-11-12T20:52:36.250ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 88063ec3-63ca-4474-96d0-cc5b2ef16180 step component = MgsUpdates step id = Ensure
153782025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Dependency
153792025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153802025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Dependency
153812025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
153822025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 974.325094ms iteration = 9 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4480,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":78942213,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":78942153,"secs":0}},"event_index":1,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":78965174,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":28466657,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":28466497,"secs":0}},"event_index":2,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107446701,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":205706474,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":205706424,"secs":0}},"event_index":3,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":313171745,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":228012340,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":228012290,"secs":0}},"event_index":4,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":541209356,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":229968301,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":229968261,"secs":0}},"event_index":5,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":771192728,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":93780217,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":93780177,"secs":0}},"event_index":6,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":864981585,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4130,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4090,"secs":0}},"event_index":7,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":864994865,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2020,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":1991,"secs":0}},"event_index":8,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":864999316,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3760,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3720,"secs":0}},"event_index":9,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":865004676,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1840,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1800,"secs":0}},"event_index":10,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":865008176,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38211239,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38211209,"secs":0}},"event_index":11,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903220675,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13888509,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13888460,"secs":0}},"event_index":12,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":917121854,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41738490,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41738460,"secs":0}},"event_index":13,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":958879135,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15131162,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15131082,"secs":0}},"event_index":14,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974019797,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5630,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5590,"secs":0}},"event_index":15,"execution_id":"2f4259d7-faa7-4012-a67b-0deba933c4e4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974032257,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
153832025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
153842025-11-12T20:52:36.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
153852025-11-12T20:52:36.252ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153862025-11-12T20:52:36.252ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153872025-11-12T20:52:36.254ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
153882025-11-12T20:52:36.254ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153892025-11-12T20:52:36.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
153902025-11-12T20:52:36.254ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
153912025-11-12T20:52:36.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
153922025-11-12T20:52:36.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
153932025-11-12T20:52:36.256ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
153942025-11-12T20:52:36.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
153952025-11-12T20:52:36.258ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153962025-11-12T20:52:36.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
153972025-11-12T20:52:36.260ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
153982025-11-12T20:52:36.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
153992025-11-12T20:52:36.262ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
154002025-11-12T20:52:36.262ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.222447ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
154012025-11-12T20:52:36.288ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(53a0c9f3-40f8-426b-999e-1a06cd803cef, 0), (a6ce4867-8d96-4d76-9bcd-36258ef2dd87, 0), (29cc4e02-8a32-42b9-91ac-c9527551a783, 0), (c370e156-55b4-4b7c-b41f-85f59dcc2f77, 0)] collection_id = 53a0c9f3-40f8-426b-999e-1a06cd803cef
154022025-11-12T20:52:36.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154032025-11-12T20:52:36.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
154042025-11-12T20:52:36.430ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 15 transaction = inventory_delete_collection
154052025-11-12T20:52:36.484ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
154062025-11-12T20:52:36.484ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 53a0c9f3-40f8-426b-999e-1a06cd803cef file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
154072025-11-12T20:52:36.484ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154082025-11-12T20:52:36.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
154092025-11-12T20:52:36.537ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(29cc4e02-8a32-42b9-91ac-c9527551a783, 0), (c370e156-55b4-4b7c-b41f-85f59dcc2f77, 0), (5b8b6c4c-fea2-4427-b744-2d0ccac60ceb, 0)]
154102025-11-12T20:52:36.538ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 33649805 } })
154112025-11-12T20:52:36.570ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 65836918 } })
154122025-11-12T20:52:36.570ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
154132025-11-12T20:52:36.570ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
154142025-11-12T20:52:36.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
154152025-11-12T20:52:36.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "683ff636-a4b8-4533-a390-09a924fc4aa7", "content-length": "476", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154162025-11-12T20:52:36.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
154172025-11-12T20:52:36.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "392f0620-a5e2-4a57-9961-2aafbb183a5e", "content-length": "734", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154182025-11-12T20:52:36.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
154192025-11-12T20:52:36.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c688609-7d6e-41f1-99d3-3b4916529ca2", "content-length": "111", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154202025-11-12T20:52:36.572ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
154212025-11-12T20:52:36.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "948978df-96df-42a1-923a-d91e2b7cc8f4", "content-length": "111", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154222025-11-12T20:52:36.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
154232025-11-12T20:52:36.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81b0931f-2f0f-428e-bcb2-fc09852bf8f2", "content-length": "163", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154242025-11-12T20:52:36.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
154252025-11-12T20:52:36.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbfa8310-a37d-44cb-9013-56f22242b27d", "content-length": "163", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154262025-11-12T20:52:36.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
154272025-11-12T20:52:36.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2142fdd3-5a42-4363-9c9a-a7c528c42495", "content-length": "166", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154282025-11-12T20:52:36.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
154292025-11-12T20:52:36.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfeefe19-1f2f-41ae-8b3b-cd46396cf064", "content-length": "166", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154302025-11-12T20:52:36.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
154312025-11-12T20:52:36.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec0f625c-18c4-43e7-b193-b29e6db5bc9b", "content-length": "702", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154322025-11-12T20:52:36.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
154332025-11-12T20:52:36.575ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78dee1bf-3dc7-4a68-99c7-e119b9d52558", "content-length": "727", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154342025-11-12T20:52:36.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
154352025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ec16eb-6554-4add-95fc-f2c7ee1dbba6", "content-length": "729", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154362025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
154372025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14641533-f640-4bce-bd0f-ef33fd70daee", "content-length": "728", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154382025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
154392025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e21ab139-7a7e-4781-8401-c7517009066c", "content-length": "722", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154402025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
154412025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dda462e-9298-461c-b67f-ac2e75e1409c", "content-length": "10", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154422025-11-12T20:52:36.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
154432025-11-12T20:52:36.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cde6fb65-653e-4909-be59-317d48688140", "content-length": "144", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154442025-11-12T20:52:36.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
154452025-11-12T20:52:36.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4698b0c-5616-4f1e-bd40-bc84bf406f32", "content-length": "147", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154462025-11-12T20:52:36.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
154472025-11-12T20:52:36.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92998b46-e1d1-4c68-890d-a6936af90de1", "content-length": "109", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154482025-11-12T20:52:36.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
154492025-11-12T20:52:36.578ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c47c51b-c3db-4c82-adb4-8aea783d295c", "content-length": "109", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154502025-11-12T20:52:36.578ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
154512025-11-12T20:52:36.578ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82931064-d792-4686-9ffb-ae960ab511f4", "content-length": "163", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154522025-11-12T20:52:36.578ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
154532025-11-12T20:52:36.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "74aee58b-c516-4c37-b45f-4768014f902c", "content-length": "163", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154542025-11-12T20:52:36.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
154552025-11-12T20:52:36.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f79dbaf-aa8d-47cc-b97f-e4f50606ce44", "content-length": "166", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154562025-11-12T20:52:36.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
154572025-11-12T20:52:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "208ba330-159d-476f-8aec-152efeffbf3d", "content-length": "166", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154582025-11-12T20:52:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
154592025-11-12T20:52:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f72eca00-e87b-4498-b1a6-ce61d43aa95f", "content-length": "702", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154602025-11-12T20:52:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
154612025-11-12T20:52:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7aaf6575-ca84-40a3-9e50-184328d3daf0", "content-length": "727", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154622025-11-12T20:52:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
154632025-11-12T20:52:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2e3c9cd-2108-4f58-b326-7e51f7aeea78", "content-length": "729", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154642025-11-12T20:52:36.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
154652025-11-12T20:52:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6074cec5-ecfc-4092-8b9b-440a9b4a08c2", "content-length": "728", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154662025-11-12T20:52:36.581ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154672025-11-12T20:52:36.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
154682025-11-12T20:52:36.583ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154692025-11-12T20:52:36.585ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
154702025-11-12T20:52:36.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154712025-11-12T20:52:36.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
154722025-11-12T20:52:36.587ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154732025-11-12T20:52:36.589ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
154742025-11-12T20:52:36.589ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154752025-11-12T20:52:36.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
154762025-11-12T20:52:36.623ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
154772025-11-12T20:52:36.623ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
154782025-11-12T20:52:36.763ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb6516e2-6930-4694-8cc7-b7ec1b30fa23", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154792025-11-12T20:52:36.765ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
154802025-11-12T20:52:36.765ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
154812025-11-12T20:52:36.765ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
154822025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
154832025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
154842025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
154852025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
154862025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
154872025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "338f7a98-4063-4510-96b6-ea0c12dc343c", "content-length": "64", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154882025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
154892025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "db09f1c4-08a5-474e-b54a-3b886da15469", "content-length": "64", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154902025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
154912025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "628958c3-d2d4-4998-9525-2d09a959e6b7", "content-length": "64", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154922025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "66f3493c-f60a-4894-8f7c-d9beacb32519", "content-length": "64", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154932025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
154942025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
154952025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b7b2598-cf3b-4c57-bf68-2ed0fdf4b556", "content-length": "64", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154962025-11-12T20:52:36.765ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
154972025-11-12T20:52:36.802ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d19c2142-8251-48cc-b7c2-816489124dca", "content-length": "398956", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154982025-11-12T20:52:36.803ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6a4df62-6fa1-45e7-b644-91a3007f146f", "content-length": "396752", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
154992025-11-12T20:52:36.803ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "967a8e36-5e5f-4ea6-9476-eed3e6ed0fa4", "content-length": "427778", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155002025-11-12T20:52:36.803ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "58398ae1-d637-40ee-9faf-09ea2370c5dc", "content-length": "399256", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155012025-11-12T20:52:36.803ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0293c3b-a77e-484b-8b70-455919a222b6", "content-length": "401290", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155022025-11-12T20:52:36.806ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
155032025-11-12T20:52:36.806ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
155042025-11-12T20:52:36.806ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
155052025-11-12T20:52:36.806ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
155062025-11-12T20:52:36.806ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
155072025-11-12T20:52:36.837ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
155082025-11-12T20:52:36.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
155092025-11-12T20:52:36.840ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "02132581-7fff-43c2-90b9-3837ca8406bf", "content-length": "126", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155102025-11-12T20:52:36.841ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
155112025-11-12T20:52:36.934ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
155122025-11-12T20:52:36.934ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
155132025-11-12T20:52:36.934ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2e572c0-426d-4662-b93b-4145c6bba922", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155142025-11-12T20:52:36.935ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
155152025-11-12T20:52:36.937ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
155162025-11-12T20:52:36.937ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
155172025-11-12T20:52:36.937ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6062d8a-2e85-4176-b6ed-276e7a1a9f7e", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155182025-11-12T20:52:36.937ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
155192025-11-12T20:52:36.939ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
155202025-11-12T20:52:36.939ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
155212025-11-12T20:52:36.940ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aa68c6e-1f2c-4ded-b1fe-ff95fc59a756", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:52:36 GMT"} })
155222025-11-12T20:52:36.940ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
155232025-11-12T20:52:36.941ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
155242025-11-12T20:52:36.941ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
155252025-11-12T20:52:36.944ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155262025-11-12T20:52:36.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
155272025-11-12T20:52:36.946ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155282025-11-12T20:52:36.949ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
155292025-11-12T20:52:36.949ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155302025-11-12T20:52:36.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
155312025-11-12T20:52:37.167ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = f1d519c6-906b-463e-a78e-1e59b98d7eac file = nexus/db-queries/src/db/datastore/inventory.rs:1614
155322025-11-12T20:52:37.167ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = f1d519c6-906b-463e-a78e-1e59b98d7eac time_started = 2025-11-12 20:52:36.570927 UTC
155332025-11-12T20:52:37.167ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 38 reason = Dependency
155342025-11-12T20:52:37.167ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
155352025-11-12T20:52:37.167ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 917.387971ms iteration = 9 status = {"collection_id":"f1d519c6-906b-463e-a78e-1e59b98d7eac","time_done":"2025-11-12 20:52:36.941429 UTC","time_started":"2025-11-12 20:52:36.570927 UTC"}
155362025-11-12T20:52:37.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
155372025-11-12T20:52:37.170ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155382025-11-12T20:52:37.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
155392025-11-12T20:52:37.174ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
155402025-11-12T20:52:37.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
155412025-11-12T20:52:37.177ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
155422025-11-12T20:52:37.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
155432025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 12 reason = Dependency
155442025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 69.476498ms iteration = 38 status = {"Loaded":{"collection_id":"49918508-77d9-410a-ad8e-292b92856858","time_loaded":"2025-11-12T20:52:37.167709760Z","time_started":"2025-11-12T20:52:36.571589Z"}}
155452025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 20 reason = Dependency
155462025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Dependency
155472025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
155482025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
155492025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155502025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
155512025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
155522025-11-12T20:52:37.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155532025-11-12T20:52:37.239ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155542025-11-12T20:52:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155552025-11-12T20:52:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155562025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155572025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155582025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155592025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
155602025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
155612025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
155622025-11-12T20:52:37.243ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
155632025-11-12T20:52:37.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
155642025-11-12T20:52:37.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
155652025-11-12T20:52:37.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155662025-11-12T20:52:37.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
155672025-11-12T20:52:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
155682025-11-12T20:52:37.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155692025-11-12T20:52:37.249ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155702025-11-12T20:52:37.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155712025-11-12T20:52:37.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155722025-11-12T20:52:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155732025-11-12T20:52:37.251ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
155742025-11-12T20:52:37.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
155752025-11-12T20:52:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
155762025-11-12T20:52:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
155772025-11-12T20:52:37.252ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
155782025-11-12T20:52:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
155792025-11-12T20:52:37.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155802025-11-12T20:52:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155812025-11-12T20:52:37.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
155822025-11-12T20:52:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
155832025-11-12T20:52:37.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155842025-11-12T20:52:37.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155852025-11-12T20:52:37.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155862025-11-12T20:52:37.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155872025-11-12T20:52:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155882025-11-12T20:52:37.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155892025-11-12T20:52:37.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
155902025-11-12T20:52:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
155912025-11-12T20:52:37.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
155922025-11-12T20:52:37.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155932025-11-12T20:52:37.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155942025-11-12T20:52:37.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155952025-11-12T20:52:37.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155962025-11-12T20:52:37.266ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
155972025-11-12T20:52:37.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
155982025-11-12T20:52:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
155992025-11-12T20:52:37.269ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
156002025-11-12T20:52:37.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
156012025-11-12T20:52:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
156022025-11-12T20:52:37.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156032025-11-12T20:52:37.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
156042025-11-12T20:52:37.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
156052025-11-12T20:52:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
156062025-11-12T20:52:37.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156072025-11-12T20:52:37.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
156082025-11-12T20:52:37.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
156092025-11-12T20:52:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
156102025-11-12T20:52:37.281ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
156112025-11-12T20:52:37.281ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 43.837855ms iteration = 12 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"49918508-77d9-410a-ad8e-292b92856858","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
156122025-11-12T20:52:37.315ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 78.344157ms iteration = 25 status = {"physical_disks_added":0}
156132025-11-12T20:52:37.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156142025-11-12T20:52:37.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156152025-11-12T20:52:37.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156162025-11-12T20:52:37.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156172025-11-12T20:52:37.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156182025-11-12T20:52:37.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
156192025-11-12T20:52:37.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156202025-11-12T20:52:37.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156212025-11-12T20:52:37.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156222025-11-12T20:52:37.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156232025-11-12T20:52:37.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156242025-11-12T20:52:37.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
156252025-11-12T20:52:37.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156262025-11-12T20:52:37.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156272025-11-12T20:52:37.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156282025-11-12T20:52:37.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
156292025-11-12T20:52:37.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156302025-11-12T20:52:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156312025-11-12T20:52:37.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156322025-11-12T20:52:37.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156332025-11-12T20:52:37.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156342025-11-12T20:52:37.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
156352025-11-12T20:52:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156362025-11-12T20:52:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156372025-11-12T20:52:37.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156382025-11-12T20:52:37.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156392025-11-12T20:52:37.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156402025-11-12T20:52:37.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
156412025-11-12T20:52:37.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156422025-11-12T20:52:37.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156432025-11-12T20:52:37.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156442025-11-12T20:52:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156452025-11-12T20:52:37.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156462025-11-12T20:52:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
156472025-11-12T20:52:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156482025-11-12T20:52:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156492025-11-12T20:52:37.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156502025-11-12T20:52:37.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156512025-11-12T20:52:37.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156522025-11-12T20:52:37.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156532025-11-12T20:52:37.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156542025-11-12T20:52:37.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
156552025-11-12T20:52:37.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156562025-11-12T20:52:37.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156572025-11-12T20:52:37.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156582025-11-12T20:52:37.396ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
156592025-11-12T20:52:37.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156602025-11-12T20:52:37.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156612025-11-12T20:52:37.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156622025-11-12T20:52:37.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156632025-11-12T20:52:37.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156642025-11-12T20:52:37.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156652025-11-12T20:52:37.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156662025-11-12T20:52:37.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
156672025-11-12T20:52:37.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156682025-11-12T20:52:37.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156692025-11-12T20:52:37.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156702025-11-12T20:52:37.410ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
156712025-11-12T20:52:37.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156722025-11-12T20:52:37.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156732025-11-12T20:52:37.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156742025-11-12T20:52:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156752025-11-12T20:52:37.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156762025-11-12T20:52:37.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156772025-11-12T20:52:37.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156782025-11-12T20:52:37.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
156792025-11-12T20:52:37.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156802025-11-12T20:52:37.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156812025-11-12T20:52:37.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156822025-11-12T20:52:37.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156832025-11-12T20:52:37.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156842025-11-12T20:52:37.426ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
156852025-11-12T20:52:37.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156862025-11-12T20:52:37.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156872025-11-12T20:52:37.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156882025-11-12T20:52:37.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156892025-11-12T20:52:37.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156902025-11-12T20:52:37.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
156912025-11-12T20:52:37.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156922025-11-12T20:52:37.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156932025-11-12T20:52:37.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156942025-11-12T20:52:37.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156952025-11-12T20:52:37.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156962025-11-12T20:52:37.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
156972025-11-12T20:52:37.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156982025-11-12T20:52:37.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156992025-11-12T20:52:37.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157002025-11-12T20:52:37.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157012025-11-12T20:52:37.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157022025-11-12T20:52:37.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
157032025-11-12T20:52:37.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157042025-11-12T20:52:37.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157052025-11-12T20:52:37.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157062025-11-12T20:52:37.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157072025-11-12T20:52:37.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157082025-11-12T20:52:37.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
157092025-11-12T20:52:37.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157102025-11-12T20:52:37.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157112025-11-12T20:52:37.485ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
157122025-11-12T20:52:37.485ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
157132025-11-12T20:52:37.485ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 04c94080-a8de-4243-b3be-425ca03a1b74 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
157142025-11-12T20:52:37.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157152025-11-12T20:52:37.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157162025-11-12T20:52:37.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157172025-11-12T20:52:37.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
157182025-11-12T20:52:37.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157192025-11-12T20:52:37.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157202025-11-12T20:52:37.493ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
157212025-11-12T20:52:37.493ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
157222025-11-12T20:52:37.493ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 256.519518ms iteration = 20 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
157232025-11-12T20:52:37.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 377 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 9330b332-8ab5-4ed4-a07d-e88ffcc5ee69 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
157242025-11-12T20:52:40.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 86 reason = Timeout
157252025-11-12T20:52:40.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157262025-11-12T20:52:40.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157272025-11-12T20:52:40.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157282025-11-12T20:52:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
157292025-11-12T20:52:40.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157302025-11-12T20:52:40.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157312025-11-12T20:52:40.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
157322025-11-12T20:52:40.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.595156ms iteration = 86 status = {"config_updated":false}
157332025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
157342025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
157352025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
157362025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 87 reason = Timeout
157372025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157382025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157392025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
157402025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157412025-11-12T20:52:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157422025-11-12T20:52:45.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
157432025-11-12T20:52:45.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157442025-11-12T20:52:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157452025-11-12T20:52:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
157462025-11-12T20:52:45.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.587753ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
157472025-11-12T20:52:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
157482025-11-12T20:52:45.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157492025-11-12T20:52:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
157502025-11-12T20:52:45.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157512025-11-12T20:52:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157522025-11-12T20:52:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157532025-11-12T20:52:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157542025-11-12T20:52:45.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
157552025-11-12T20:52:45.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.131485ms iteration = 87 status = {"config_updated":false}
157562025-11-12T20:52:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157572025-11-12T20:52:45.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157582025-11-12T20:52:45.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
157592025-11-12T20:52:45.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157602025-11-12T20:52:45.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157612025-11-12T20:52:45.301ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
157622025-11-12T20:52:45.301ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.460387ms iteration = 44 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
157632025-11-12T20:52:47.915ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 5455842d-1680-470e-a3d8-0c6a3dd2495a uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
157642025-11-12T20:52:47.915ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 5455842d-1680-470e-a3d8-0c6a3dd2495a roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
157652025-11-12T20:52:47.916ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 5455842d-1680-470e-a3d8-0c6a3dd2495a resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
157662025-11-12T20:52:47.918ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 5455842d-1680-470e-a3d8-0c6a3dd2495a response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
157672025-11-12T20:52:47.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 10bb2032-2c36-493b-b58a-fa09920e4f91 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
157682025-11-12T20:52:47.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 10bb2032-2c36-493b-b58a-fa09920e4f91 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
157692025-11-12T20:52:47.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 10bb2032-2c36-493b-b58a-fa09920e4f91 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
157702025-11-12T20:52:47.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2169 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:51395 req_id = 10bb2032-2c36-493b-b58a-fa09920e4f91 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
157712025-11-12T20:52:47.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 267 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = f0c1234a-c2fe-44a3-b97d-57511a3ac063 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
157722025-11-12T20:52:50.278ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 39 reason = Timeout
157732025-11-12T20:52:50.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
157742025-11-12T20:52:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
157752025-11-12T20:52:50.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 88 reason = Timeout
157762025-11-12T20:52:50.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157772025-11-12T20:52:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157782025-11-12T20:52:50.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157792025-11-12T20:52:50.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157802025-11-12T20:52:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
157812025-11-12T20:52:50.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
157822025-11-12T20:52:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
157832025-11-12T20:52:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
157842025-11-12T20:52:50.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157852025-11-12T20:52:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157862025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49918508-77d9-410a-ad8e-292b92856858 original_time_started = 2025-11-12 20:52:36.571589 UTC
157872025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.660599ms iteration = 39 status = {"Loaded":{"collection_id":"49918508-77d9-410a-ad8e-292b92856858","time_loaded":"2025-11-12T20:52:50.277999084Z","time_started":"2025-11-12T20:52:36.571589Z"}}
157882025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 30 reason = Timeout
157892025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
157902025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
157912025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.32736ms iteration = 88 status = {"config_updated":false}
157922025-11-12T20:52:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
157932025-11-12T20:52:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157942025-11-12T20:52:50.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
157952025-11-12T20:52:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
157962025-11-12T20:52:50.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
157972025-11-12T20:52:50.291ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.468852ms iteration = 30 status = "NoSitrep"
157982025-11-12T20:52:50.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 30 reason = Timeout
157992025-11-12T20:52:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
158002025-11-12T20:52:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
158012025-11-12T20:52:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
158022025-11-12T20:52:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
158032025-11-12T20:52:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158042025-11-12T20:52:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
158052025-11-12T20:52:50.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
158062025-11-12T20:52:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
158072025-11-12T20:52:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
158082025-11-12T20:52:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
158092025-11-12T20:52:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158102025-11-12T20:52:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
158112025-11-12T20:52:50.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
158122025-11-12T20:52:50.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.151027ms iteration = 30 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
158132025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
158142025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
158152025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
158162025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 89 reason = Timeout
158172025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158182025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
158192025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
158202025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
158212025-11-12T20:52:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
158222025-11-12T20:52:55.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158232025-11-12T20:52:55.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
158242025-11-12T20:52:55.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158252025-11-12T20:52:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
158262025-11-12T20:52:55.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.651074ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
158272025-11-12T20:52:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
158282025-11-12T20:52:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158292025-11-12T20:52:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
158302025-11-12T20:52:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
158312025-11-12T20:52:55.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
158322025-11-12T20:52:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
158332025-11-12T20:52:55.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158342025-11-12T20:52:55.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
158352025-11-12T20:52:55.287ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
158362025-11-12T20:52:55.287ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.832378ms iteration = 89 status = {"config_updated":false}
158372025-11-12T20:52:55.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158382025-11-12T20:52:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
158392025-11-12T20:52:55.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158402025-11-12T20:52:55.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
158412025-11-12T20:52:55.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
158422025-11-12T20:52:55.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.883774ms iteration = 45 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
158432025-11-12T20:52:57.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 442 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = a5058c57-b33b-431d-8e6d-7156033cd3ad response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
158442025-11-12T20:53:00.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 90 reason = Timeout
158452025-11-12T20:53:00.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
158462025-11-12T20:53:00.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
158472025-11-12T20:53:00.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158482025-11-12T20:53:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
158492025-11-12T20:53:00.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
158502025-11-12T20:53:00.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
158512025-11-12T20:53:00.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
158522025-11-12T20:53:00.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.175016ms iteration = 90 status = {"config_updated":false}
158532025-11-12T20:53:05.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
158542025-11-12T20:53:05.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 16 reason = Timeout
158552025-11-12T20:53:05.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
158562025-11-12T20:53:05.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
158572025-11-12T20:53:05.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
158582025-11-12T20:53:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
158592025-11-12T20:53:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
158602025-11-12T20:53:05.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 40 reason = Timeout
158612025-11-12T20:53:05.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
158622025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
158632025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
158642025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
158652025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
158662025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
158672025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
158682025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
158692025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
158702025-11-12T20:53:05.278ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
158712025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
158722025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158732025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
158742025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
158752025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 91 reason = Timeout
158762025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
158772025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158782025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
158792025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
158802025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
158812025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
158822025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
158832025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
158842025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
158852025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
158862025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
158872025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 16 reason = Timeout
158882025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
158892025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
158902025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
158912025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
158922025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
158932025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
158942025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
158952025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
158962025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
158972025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
158982025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
158992025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
159002025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
159012025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
159022025-11-12T20:53:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
159032025-11-12T20:53:05.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1380, tv_nsec: 774727642 } })
159042025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159052025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
159062025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
159072025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
159082025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
159092025-11-12T20:53:05.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1380, tv_nsec: 774857276 } })
159102025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
159112025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
159122025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
159132025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
159142025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
159152025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
159162025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
159172025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
159182025-11-12T20:53:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
159192025-11-12T20:53:05.280ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1380, tv_nsec: 775841941 } })
159202025-11-12T20:53:05.280ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
159212025-11-12T20:53:05.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159222025-11-12T20:53:05.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
159232025-11-12T20:53:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
159242025-11-12T20:53:05.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159252025-11-12T20:53:05.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159262025-11-12T20:53:05.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159272025-11-12T20:53:05.282ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159282025-11-12T20:53:05.284ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.637055ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
159292025-11-12T20:53:05.285ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.830085ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
159302025-11-12T20:53:05.286ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
159312025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
159322025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
159332025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 31 reason = Timeout
159342025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
159352025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159362025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
159372025-11-12T20:53:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
159382025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
159392025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
159402025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
159412025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
159422025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
159432025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
159442025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
159452025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
159462025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
159472025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
159482025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
159492025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
159502025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
159512025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
159522025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
159532025-11-12T20:53:05.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
159542025-11-12T20:53:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
159552025-11-12T20:53:05.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
159562025-11-12T20:53:05.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159572025-11-12T20:53:05.290ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
159582025-11-12T20:53:05.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
159592025-11-12T20:53:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159602025-11-12T20:53:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
159612025-11-12T20:53:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
159622025-11-12T20:53:05.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 31 reason = Timeout
159632025-11-12T20:53:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
159642025-11-12T20:53:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
159652025-11-12T20:53:05.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
159662025-11-12T20:53:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
159672025-11-12T20:53:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
159682025-11-12T20:53:05.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159692025-11-12T20:53:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
159702025-11-12T20:53:05.295ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159712025-11-12T20:53:05.296ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159722025-11-12T20:53:05.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159732025-11-12T20:53:05.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159742025-11-12T20:53:05.297ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
159752025-11-12T20:53:05.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159762025-11-12T20:53:05.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
159772025-11-12T20:53:05.299ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49918508-77d9-410a-ad8e-292b92856858 original_time_started = 2025-11-12 20:52:36.571589 UTC
159782025-11-12T20:53:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 22.04277ms iteration = 40 status = {"Loaded":{"collection_id":"49918508-77d9-410a-ad8e-292b92856858","time_loaded":"2025-11-12T20:53:05.277736342Z","time_started":"2025-11-12T20:52:36.571589Z"}}
159792025-11-12T20:53:05.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159802025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
159812025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
159822025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
159832025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
159842025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
159852025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
159862025-11-12T20:53:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
159872025-11-12T20:53:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
159882025-11-12T20:53:05.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
159892025-11-12T20:53:05.302ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
159902025-11-12T20:53:05.302ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.906899ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
159912025-11-12T20:53:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
159922025-11-12T20:53:05.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
159932025-11-12T20:53:05.303ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
159942025-11-12T20:53:05.303ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
159952025-11-12T20:53:05.303ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 25.186311ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
159962025-11-12T20:53:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
159972025-11-12T20:53:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
159982025-11-12T20:53:05.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
159992025-11-12T20:53:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
160002025-11-12T20:53:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.011664ms iteration = 16 status = 0
160012025-11-12T20:53:05.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
160022025-11-12T20:53:05.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160032025-11-12T20:53:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160042025-11-12T20:53:05.306ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
160052025-11-12T20:53:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
160062025-11-12T20:53:05.307ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 20.856579ms iteration = 31 status = "NoSitrep"
160072025-11-12T20:53:05.309ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
160082025-11-12T20:53:05.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
160092025-11-12T20:53:05.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
160102025-11-12T20:53:05.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
160112025-11-12T20:53:05.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160122025-11-12T20:53:05.321ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160132025-11-12T20:53:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
160142025-11-12T20:53:05.323ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
160152025-11-12T20:53:05.323ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 31.833494ms iteration = 31 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
160162025-11-12T20:53:05.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
160172025-11-12T20:53:05.326ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
160182025-11-12T20:53:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
160192025-11-12T20:53:05.329ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160202025-11-12T20:53:05.331ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
160212025-11-12T20:53:05.331ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.236188ms iteration = 46 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
160222025-11-12T20:53:05.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
160232025-11-12T20:53:05.332ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
160242025-11-12T20:53:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
160252025-11-12T20:53:05.351ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.579584ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
160262025-11-12T20:53:05.353ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160272025-11-12T20:53:05.355ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 77.076344ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
160282025-11-12T20:53:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
160292025-11-12T20:53:05.359ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
160302025-11-12T20:53:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
160312025-11-12T20:53:05.361ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
160322025-11-12T20:53:05.361ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 83.165992ms iteration = 91 status = {"config_updated":false}
160332025-11-12T20:53:05.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.45562ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
160342025-11-12T20:53:05.365ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
160352025-11-12T20:53:05.365ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
160362025-11-12T20:53:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
160372025-11-12T20:53:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
160382025-11-12T20:53:05.366ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160392025-11-12T20:53:05.368ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
160402025-11-12T20:53:05.368ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 89.949338ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
160412025-11-12T20:53:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
160422025-11-12T20:53:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
160432025-11-12T20:53:05.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160442025-11-12T20:53:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160452025-11-12T20:53:05.370ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
160462025-11-12T20:53:05.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
160472025-11-12T20:53:05.373ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
160482025-11-12T20:53:05.373ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
160492025-11-12T20:53:05.373ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
160502025-11-12T20:53:05.373ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1380, tv_nsec: 869357131 } })
160512025-11-12T20:53:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
160522025-11-12T20:53:05.374ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
160532025-11-12T20:53:05.411ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 132.53407ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
160542025-11-12T20:53:05.420ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 142.047106ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
160552025-11-12T20:53:05.425ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
160562025-11-12T20:53:05.425ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
160572025-11-12T20:53:05.425ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 146.557173ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
160582025-11-12T20:53:05.444ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
160592025-11-12T20:53:05.444ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "77dc840b-847a-40bd-aec8-126c1551772f", "content-length": "78", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
160602025-11-12T20:53:05.444ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
160612025-11-12T20:53:05.444ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
160622025-11-12T20:53:05.444ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
160632025-11-12T20:53:05.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
160642025-11-12T20:53:05.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
160652025-11-12T20:53:05.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
160662025-11-12T20:53:05.446ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
160672025-11-12T20:53:05.447ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
160682025-11-12T20:53:05.447ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
160692025-11-12T20:53:05.447ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
160702025-11-12T20:53:05.448ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 172.650198ms iteration = 16 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
160712025-11-12T20:53:05.450ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
160722025-11-12T20:53:05.450ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
160732025-11-12T20:53:05.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
160742025-11-12T20:53:05.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
160752025-11-12T20:53:05.451ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
160762025-11-12T20:53:05.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
160772025-11-12T20:53:05.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
160782025-11-12T20:53:05.451ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
160792025-11-12T20:53:05.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
160802025-11-12T20:53:05.452ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
160812025-11-12T20:53:05.452ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
160822025-11-12T20:53:05.452ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3ed680e-2bcc-43c0-9fd7-dcfe05f84acc", "content-length": "81", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
160832025-11-12T20:53:05.453ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
160842025-11-12T20:53:05.453ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
160852025-11-12T20:53:05.453ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
160862025-11-12T20:53:05.453ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
160872025-11-12T20:53:05.453ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "84d4d7da-e5b4-44d2-ab95-c6150b08bdc5", "content-length": "81", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
160882025-11-12T20:53:05.453ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "79206ffa-90de-4919-a9dc-1086a8e30cb1", "content-length": "81", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
160892025-11-12T20:53:05.455ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 176.920078ms iteration = 16 status = {"disabled":false,"errors":[],"sps":[]}
160902025-11-12T20:53:05.465ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 186.464356ms iteration = 26 status = {"physical_disks_added":0}
160912025-11-12T20:53:05.550ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
160922025-11-12T20:53:05.550ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
160932025-11-12T20:53:05.551ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b07d5a3b-0cef-424e-a3dd-de2552560873", "content-length": "372", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
160942025-11-12T20:53:05.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160952025-11-12T20:53:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160962025-11-12T20:53:05.553ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
160972025-11-12T20:53:05.553ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
160982025-11-12T20:53:05.553ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a03cb03-0097-4ebf-8c4d-5595cb143436", "content-length": "26", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
160992025-11-12T20:53:05.553ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
161002025-11-12T20:53:05.554ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
161012025-11-12T20:53:05.565ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
161022025-11-12T20:53:05.565ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
161032025-11-12T20:53:05.565ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8b1afed2-24f6-4d66-bd10-4828752eb9b1", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
161042025-11-12T20:53:05.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
161052025-11-12T20:53:05.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
161062025-11-12T20:53:05.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
161072025-11-12T20:53:05.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
161082025-11-12T20:53:05.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
161092025-11-12T20:53:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
161102025-11-12T20:53:05.574ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
161112025-11-12T20:53:05.574ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
161122025-11-12T20:53:05.574ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "32f4ddee-3656-4785-b318-d0138e6a4cb3", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
161132025-11-12T20:53:05.576ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 297.650693ms iteration = 24 status = {}
161142025-11-12T20:53:05.596ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
161152025-11-12T20:53:05.596ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "484bf548-490c-4c69-9aa4-f5f63bdf70eb", "content-length": "691", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
161162025-11-12T20:53:05.596ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
161172025-11-12T20:53:05.596ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
161182025-11-12T20:53:05.596ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
161192025-11-12T20:53:05.598ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 319.763035ms iteration = 16 status = {}
161202025-11-12T20:53:05.603ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
161212025-11-12T20:53:05.603ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
161222025-11-12T20:53:05.603ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "74a79142-3c0a-4250-9108-0521a1fe505e", "content-length": "26", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
161232025-11-12T20:53:05.603ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
161242025-11-12T20:53:05.606ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
161252025-11-12T20:53:05.633ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 357.481091ms iteration = 16 status = {}
161262025-11-12T20:53:05.650ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 145870667 } })
161272025-11-12T20:53:05.712ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161282025-11-12T20:53:05.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161292025-11-12T20:53:05.714ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
161302025-11-12T20:53:05.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161312025-11-12T20:53:05.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161322025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
161332025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
161342025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
161352025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
161362025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
161372025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
161382025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
161392025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
161402025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
161412025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
161422025-11-12T20:53:05.727ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
161432025-11-12T20:53:05.727ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161442025-11-12T20:53:05.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161452025-11-12T20:53:05.730ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
161462025-11-12T20:53:05.730ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
161472025-11-12T20:53:05.841ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e5325111-dbd2-4c5f-8f98-599f18c72b3f", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
161482025-11-12T20:53:05.841ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
161492025-11-12T20:53:05.841ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
161502025-11-12T20:53:05.841ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "68e96060-3073-4ea0-88f9-b9a5496401e5", "content-length": "682", "date": "Wed, 12 Nov 2025 20:53:05 GMT"} })
161512025-11-12T20:53:05.841ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161522025-11-12T20:53:05.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161532025-11-12T20:53:05.844ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161542025-11-12T20:53:05.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161552025-11-12T20:53:05.848ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161562025-11-12T20:53:05.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161572025-11-12T20:53:05.850ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
161582025-11-12T20:53:05.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
161592025-11-12T20:53:05.857ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 579.515547ms iteration = 16 status = {}
161602025-11-12T20:53:07.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 450 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 405730c3-6c46-4431-aa1e-5dc62c0885b3 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
161612025-11-12T20:53:10.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 92 reason = Timeout
161622025-11-12T20:53:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
161632025-11-12T20:53:10.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
161642025-11-12T20:53:10.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161652025-11-12T20:53:10.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
161662025-11-12T20:53:10.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
161672025-11-12T20:53:10.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
161682025-11-12T20:53:10.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
161692025-11-12T20:53:10.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.60311ms iteration = 92 status = {"config_updated":false}
161702025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
161712025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
161722025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
161732025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 93 reason = Timeout
161742025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161752025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
161762025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
161772025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
161782025-11-12T20:53:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
161792025-11-12T20:53:15.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161802025-11-12T20:53:15.281ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
161812025-11-12T20:53:15.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161822025-11-12T20:53:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
161832025-11-12T20:53:15.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.903221ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
161842025-11-12T20:53:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
161852025-11-12T20:53:15.285ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
161862025-11-12T20:53:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
161872025-11-12T20:53:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
161882025-11-12T20:53:15.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161892025-11-12T20:53:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
161902025-11-12T20:53:15.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
161912025-11-12T20:53:15.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.195726ms iteration = 93 status = {"config_updated":false}
161922025-11-12T20:53:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161932025-11-12T20:53:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
161942025-11-12T20:53:15.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161952025-11-12T20:53:15.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
161962025-11-12T20:53:15.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161972025-11-12T20:53:15.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
161982025-11-12T20:53:15.307ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
161992025-11-12T20:53:15.307ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.080347ms iteration = 47 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
162002025-11-12T20:53:17.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 467 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = df26e7b0-b091-4fe1-b32c-f8969b699916 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
162012025-11-12T20:53:20.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 41 reason = Timeout
162022025-11-12T20:53:20.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
162032025-11-12T20:53:20.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
162042025-11-12T20:53:20.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 94 reason = Timeout
162052025-11-12T20:53:20.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
162062025-11-12T20:53:20.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
162072025-11-12T20:53:20.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162082025-11-12T20:53:20.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162092025-11-12T20:53:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
162102025-11-12T20:53:20.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
162112025-11-12T20:53:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
162122025-11-12T20:53:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
162132025-11-12T20:53:20.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
162142025-11-12T20:53:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
162152025-11-12T20:53:20.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49918508-77d9-410a-ad8e-292b92856858 original_time_started = 2025-11-12 20:52:36.571589 UTC
162162025-11-12T20:53:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.744131ms iteration = 41 status = {"Loaded":{"collection_id":"49918508-77d9-410a-ad8e-292b92856858","time_loaded":"2025-11-12T20:53:20.277832157Z","time_started":"2025-11-12T20:52:36.571589Z"}}
162172025-11-12T20:53:20.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
162182025-11-12T20:53:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.227987ms iteration = 94 status = {"config_updated":false}
162192025-11-12T20:53:20.286ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 32 reason = Timeout
162202025-11-12T20:53:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
162212025-11-12T20:53:20.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
162222025-11-12T20:53:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162232025-11-12T20:53:20.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
162242025-11-12T20:53:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
162252025-11-12T20:53:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
162262025-11-12T20:53:20.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 32 reason = Timeout
162272025-11-12T20:53:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
162282025-11-12T20:53:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
162292025-11-12T20:53:20.292ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.867981ms iteration = 32 status = "NoSitrep"
162302025-11-12T20:53:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
162312025-11-12T20:53:20.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
162322025-11-12T20:53:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162332025-11-12T20:53:20.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
162342025-11-12T20:53:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
162352025-11-12T20:53:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
162362025-11-12T20:53:20.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
162372025-11-12T20:53:20.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
162382025-11-12T20:53:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162392025-11-12T20:53:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
162402025-11-12T20:53:20.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
162412025-11-12T20:53:20.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.591198ms iteration = 32 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
162422025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
162432025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
162442025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
162452025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 95 reason = Timeout
162462025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
162472025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
162482025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
162492025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
162502025-11-12T20:53:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
162512025-11-12T20:53:25.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162522025-11-12T20:53:25.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162532025-11-12T20:53:25.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
162542025-11-12T20:53:25.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
162552025-11-12T20:53:25.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.011184ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
162562025-11-12T20:53:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
162572025-11-12T20:53:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
162582025-11-12T20:53:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
162592025-11-12T20:53:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
162602025-11-12T20:53:25.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
162612025-11-12T20:53:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
162622025-11-12T20:53:25.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
162632025-11-12T20:53:25.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
162642025-11-12T20:53:25.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
162652025-11-12T20:53:25.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.844463ms iteration = 95 status = {"config_updated":false}
162662025-11-12T20:53:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162672025-11-12T20:53:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
162682025-11-12T20:53:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
162692025-11-12T20:53:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
162702025-11-12T20:53:25.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
162712025-11-12T20:53:25.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.885561ms iteration = 48 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
162722025-11-12T20:53:27.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 326 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 2840b6b3-5cf3-41f9-9c9b-45fca5405e63 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
162732025-11-12T20:53:30.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 96 reason = Timeout
162742025-11-12T20:53:30.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
162752025-11-12T20:53:30.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
162762025-11-12T20:53:30.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162772025-11-12T20:53:30.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
162782025-11-12T20:53:30.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
162792025-11-12T20:53:30.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
162802025-11-12T20:53:30.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
162812025-11-12T20:53:30.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.005422ms iteration = 96 status = {"config_updated":false}
162822025-11-12T20:53:32.915ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:36373
162832025-11-12T20:53:32.915ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 7a15b6fb-d1da-495e-98ff-f10284f16f31 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
162842025-11-12T20:53:32.916ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 7a15b6fb-d1da-495e-98ff-f10284f16f31 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
162852025-11-12T20:53:32.916ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 7a15b6fb-d1da-495e-98ff-f10284f16f31 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
162862025-11-12T20:53:32.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3264 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 7a15b6fb-d1da-495e-98ff-f10284f16f31 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
162872025-11-12T20:53:32.919ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 9626dec8-ad0f-4099-ba7d-01dc3932c76e uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
162882025-11-12T20:53:32.919ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 9626dec8-ad0f-4099-ba7d-01dc3932c76e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
162892025-11-12T20:53:32.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 9626dec8-ad0f-4099-ba7d-01dc3932c76e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
162902025-11-12T20:53:32.922ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2799 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36373 req_id = 9626dec8-ad0f-4099-ba7d-01dc3932c76e response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
162912025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
162922025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
162932025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
162942025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
162952025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
162962025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
162972025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
162982025-11-12T20:53:35.275ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
162992025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
163002025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
163012025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
163022025-11-12T20:53:35.275ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = ExternalNetworking step id = Ensure
163032025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163042025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
163052025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
163062025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
163072025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
163082025-11-12T20:53:35.275ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:43:35.275681189 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
163092025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
163102025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
163112025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
163122025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
163132025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
163142025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
163152025-11-12T20:53:35.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
163162025-11-12T20:53:35.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 771871235 } })
163172025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
163182025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163192025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
163202025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
163212025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
163222025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
163232025-11-12T20:53:35.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 772105921 } })
163242025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
163252025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
163262025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.114739ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
163272025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
163282025-11-12T20:53:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.263143ms iteration = 9 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
163292025-11-12T20:53:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
163302025-11-12T20:53:35.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 42 reason = Timeout
163312025-11-12T20:53:35.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
163322025-11-12T20:53:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
163332025-11-12T20:53:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
163342025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
163352025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
163362025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
163372025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
163382025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
163392025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
163402025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
163412025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
163422025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
163432025-11-12T20:53:35.278ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
163442025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
163452025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 21 reason = Timeout
163462025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
163472025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
163482025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
163492025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
163502025-11-12T20:53:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
163512025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
163522025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
163532025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
163542025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
163552025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
163562025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
163572025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
163582025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
163592025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
163602025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
163612025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
163622025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
163632025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
163642025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
163652025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
163662025-11-12T20:53:35.279ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
163672025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
163682025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
163692025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
163702025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
163712025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
163722025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
163732025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
163742025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
163752025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
163762025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
163772025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
163782025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
163792025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
163802025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
163812025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
163822025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
163832025-11-12T20:53:35.279ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
163842025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
163852025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
163862025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
163872025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
163882025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
163892025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
163902025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
163912025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
163922025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
163932025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
163942025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 97 reason = Timeout
163952025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
163962025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
163972025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
163982025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
163992025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
164002025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
164012025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
164022025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
164032025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
164042025-11-12T20:53:35.280ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 775749266 } })
164052025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164062025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
164072025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
164082025-11-12T20:53:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
164092025-11-12T20:53:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
164102025-11-12T20:53:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
164112025-11-12T20:53:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
164122025-11-12T20:53:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
164132025-11-12T20:53:35.281ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 776578847 } })
164142025-11-12T20:53:35.281ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:43:35.275681189 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
164152025-11-12T20:53:35.281ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.386969ms iteration = 9 status = {"expiration":"2025-11-12T20:43:35.275681189Z","pruned":[]}
164162025-11-12T20:53:35.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
164172025-11-12T20:53:35.282ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164182025-11-12T20:53:35.283ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
164192025-11-12T20:53:35.284ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.817384ms iteration = 9 status = {"by_rx":{},"error":null}
164202025-11-12T20:53:35.284ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.677045ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
164212025-11-12T20:53:35.284ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 780087908 } })
164222025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164232025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
164242025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164252025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 33 reason = Timeout
164262025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
164272025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
164282025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
164292025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.784425ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
164302025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
164312025-11-12T20:53:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
164322025-11-12T20:53:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
164332025-11-12T20:53:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
164342025-11-12T20:53:35.286ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.641712ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
164352025-11-12T20:53:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
164362025-11-12T20:53:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164372025-11-12T20:53:35.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164382025-11-12T20:53:35.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164392025-11-12T20:53:35.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
164402025-11-12T20:53:35.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
164412025-11-12T20:53:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
164422025-11-12T20:53:35.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
164432025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
164442025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
164452025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
164462025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 9 reason = Timeout
164472025-11-12T20:53:35.291ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
164482025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 33 reason = Timeout
164492025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
164502025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
164512025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
164522025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164532025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164542025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164552025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
164562025-11-12T20:53:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164572025-11-12T20:53:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
164582025-11-12T20:53:35.292ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
164592025-11-12T20:53:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
164602025-11-12T20:53:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
164612025-11-12T20:53:35.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164622025-11-12T20:53:35.293ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
164632025-11-12T20:53:35.293ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
164642025-11-12T20:53:35.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
164652025-11-12T20:53:35.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164662025-11-12T20:53:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
164672025-11-12T20:53:35.294ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164682025-11-12T20:53:35.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164692025-11-12T20:53:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
164702025-11-12T20:53:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
164712025-11-12T20:53:35.296ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
164722025-11-12T20:53:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
164732025-11-12T20:53:35.297ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
164742025-11-12T20:53:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
164752025-11-12T20:53:35.298ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
164762025-11-12T20:53:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
164772025-11-12T20:53:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
164782025-11-12T20:53:35.298ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
164792025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164802025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
164812025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164822025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164832025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
164842025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
164852025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
164862025-11-12T20:53:35.299ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
164872025-11-12T20:53:35.300ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
164882025-11-12T20:53:35.300ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
164892025-11-12T20:53:35.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
164902025-11-12T20:53:35.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
164912025-11-12T20:53:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
164922025-11-12T20:53:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
164932025-11-12T20:53:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
164942025-11-12T20:53:35.301ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
164952025-11-12T20:53:35.301ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
164962025-11-12T20:53:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
164972025-11-12T20:53:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
164982025-11-12T20:53:35.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164992025-11-12T20:53:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
165002025-11-12T20:53:35.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
165012025-11-12T20:53:35.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165022025-11-12T20:53:35.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165032025-11-12T20:53:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
165042025-11-12T20:53:35.304ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165052025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
165062025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
165072025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
165082025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165092025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
165102025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
165112025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165122025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 26.621869ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
165132025-11-12T20:53:35.305ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
165142025-11-12T20:53:35.306ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
165152025-11-12T20:53:35.306ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
165162025-11-12T20:53:35.306ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
165172025-11-12T20:53:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
165182025-11-12T20:53:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
165192025-11-12T20:53:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
165202025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
165212025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165222025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
165232025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165242025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
165252025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165262025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
165272025-11-12T20:53:35.307ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 28.325983ms iteration = 97 status = {"config_updated":false}
165282025-11-12T20:53:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
165292025-11-12T20:53:35.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
165302025-11-12T20:53:35.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
165312025-11-12T20:53:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
165322025-11-12T20:53:35.308ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165332025-11-12T20:53:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
165342025-11-12T20:53:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165352025-11-12T20:53:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
165362025-11-12T20:53:35.311ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
165372025-11-12T20:53:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
165382025-11-12T20:53:35.312ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165392025-11-12T20:53:35.312ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 26.992188ms iteration = 33 status = "NoSitrep"
165402025-11-12T20:53:35.312ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1410, tv_nsec: 808052202 } })
165412025-11-12T20:53:35.313ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
165422025-11-12T20:53:35.313ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 34.525334ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
165432025-11-12T20:53:35.313ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165442025-11-12T20:53:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
165452025-11-12T20:53:35.314ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
165462025-11-12T20:53:35.314ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 49918508-77d9-410a-ad8e-292b92856858 original_time_started = 2025-11-12 20:52:36.571589 UTC
165472025-11-12T20:53:35.314ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 37.205403ms iteration = 42 status = {"Loaded":{"collection_id":"49918508-77d9-410a-ad8e-292b92856858","time_loaded":"2025-11-12T20:53:35.277434344Z","time_started":"2025-11-12T20:52:36.571589Z"}}
165482025-11-12T20:53:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
165492025-11-12T20:53:35.314ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165502025-11-12T20:53:35.315ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
165512025-11-12T20:53:35.315ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 35.709124ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
165522025-11-12T20:53:35.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165532025-11-12T20:53:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165542025-11-12T20:53:35.316ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
165552025-11-12T20:53:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
165562025-11-12T20:53:35.318ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
165572025-11-12T20:53:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165582025-11-12T20:53:35.318ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
165592025-11-12T20:53:35.318ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
165602025-11-12T20:53:35.318ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 40.288433ms iteration = 17 status = 0
165612025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
165622025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
165632025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
165642025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
165652025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
165662025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
165672025-11-12T20:53:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
165682025-11-12T20:53:35.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
165692025-11-12T20:53:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
165702025-11-12T20:53:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
165712025-11-12T20:53:35.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
165722025-11-12T20:53:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
165732025-11-12T20:53:35.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165742025-11-12T20:53:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165752025-11-12T20:53:35.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
165762025-11-12T20:53:35.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
165772025-11-12T20:53:35.333ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
165782025-11-12T20:53:35.334ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165792025-11-12T20:53:35.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165802025-11-12T20:53:35.336ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
165812025-11-12T20:53:35.336ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 57.533708ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
165822025-11-12T20:53:35.337ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
165832025-11-12T20:53:35.338ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
165842025-11-12T20:53:35.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
165852025-11-12T20:53:35.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165862025-11-12T20:53:35.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
165872025-11-12T20:53:35.339ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
165882025-11-12T20:53:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
165892025-11-12T20:53:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
165902025-11-12T20:53:35.340ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165912025-11-12T20:53:35.340ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165922025-11-12T20:53:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165932025-11-12T20:53:35.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
165942025-11-12T20:53:35.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165952025-11-12T20:53:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165962025-11-12T20:53:35.344ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
165972025-11-12T20:53:35.344ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
165982025-11-12T20:53:35.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
165992025-11-12T20:53:35.345ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
166002025-11-12T20:53:35.349ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
166012025-11-12T20:53:35.352ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166022025-11-12T20:53:35.354ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
166032025-11-12T20:53:35.357ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
166042025-11-12T20:53:35.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166052025-11-12T20:53:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
166062025-11-12T20:53:35.361ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
166072025-11-12T20:53:35.361ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 69.93183ms iteration = 33 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
166082025-11-12T20:53:35.361ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
166092025-11-12T20:53:35.361ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
166102025-11-12T20:53:35.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166112025-11-12T20:53:35.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
166122025-11-12T20:53:35.362ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166132025-11-12T20:53:35.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166142025-11-12T20:53:35.362ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
166152025-11-12T20:53:35.364ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166162025-11-12T20:53:35.364ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
166172025-11-12T20:53:35.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
166182025-11-12T20:53:35.366ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
166192025-11-12T20:53:35.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
166202025-11-12T20:53:35.367ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
166212025-11-12T20:53:35.368ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
166222025-11-12T20:53:35.368ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
166232025-11-12T20:53:35.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166242025-11-12T20:53:35.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
166252025-11-12T20:53:35.369ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
166262025-11-12T20:53:35.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166272025-11-12T20:53:35.370ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = SledList step id = Fetch
166282025-11-12T20:53:35.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
166292025-11-12T20:53:35.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
166302025-11-12T20:53:35.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166312025-11-12T20:53:35.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166322025-11-12T20:53:35.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166332025-11-12T20:53:35.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
166342025-11-12T20:53:35.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
166352025-11-12T20:53:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
166362025-11-12T20:53:35.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166372025-11-12T20:53:35.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166382025-11-12T20:53:35.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166392025-11-12T20:53:35.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166402025-11-12T20:53:35.385ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
166412025-11-12T20:53:35.385ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
166422025-11-12T20:53:35.385ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 106.336641ms iteration = 49 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
166432025-11-12T20:53:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
166442025-11-12T20:53:35.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166452025-11-12T20:53:35.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166462025-11-12T20:53:35.388ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
166472025-11-12T20:53:35.388ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
166482025-11-12T20:53:35.388ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
166492025-11-12T20:53:35.388ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
166502025-11-12T20:53:35.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
166512025-11-12T20:53:35.388ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166522025-11-12T20:53:35.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166532025-11-12T20:53:35.391ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
166542025-11-12T20:53:35.391ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
166552025-11-12T20:53:35.391ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
166562025-11-12T20:53:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
166572025-11-12T20:53:35.392ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166582025-11-12T20:53:35.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166592025-11-12T20:53:35.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166602025-11-12T20:53:35.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 117.321286ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
166612025-11-12T20:53:35.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 118.168678ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
166622025-11-12T20:53:35.396ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166632025-11-12T20:53:35.397ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
166642025-11-12T20:53:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
166652025-11-12T20:53:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
166662025-11-12T20:53:35.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166672025-11-12T20:53:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
166682025-11-12T20:53:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166692025-11-12T20:53:35.399ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
166702025-11-12T20:53:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
166712025-11-12T20:53:35.400ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
166722025-11-12T20:53:35.400ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
166732025-11-12T20:53:35.400ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 121.636897ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
166742025-11-12T20:53:35.401ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
166752025-11-12T20:53:35.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
166762025-11-12T20:53:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
166772025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
166782025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166792025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
166802025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
166812025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
166822025-11-12T20:53:35.402ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
166832025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 123.64319ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
166842025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
166852025-11-12T20:53:35.402ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 123.886126ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
166862025-11-12T20:53:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
166872025-11-12T20:53:35.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
166882025-11-12T20:53:35.404ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166892025-11-12T20:53:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
166902025-11-12T20:53:35.405ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
166912025-11-12T20:53:35.405ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = DeployNexusRecords step id = Ensure
166922025-11-12T20:53:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
166932025-11-12T20:53:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
166942025-11-12T20:53:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
166952025-11-12T20:53:35.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166962025-11-12T20:53:35.407ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
166972025-11-12T20:53:35.407ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166982025-11-12T20:53:35.407ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
166992025-11-12T20:53:35.407ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
167002025-11-12T20:53:35.407ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 132.061297ms iteration = 9 status = {"generation":1}
167012025-11-12T20:53:35.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
167022025-11-12T20:53:35.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167032025-11-12T20:53:35.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167042025-11-12T20:53:35.409ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
167052025-11-12T20:53:35.409ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 130.303592ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
167062025-11-12T20:53:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
167072025-11-12T20:53:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
167082025-11-12T20:53:35.411ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
167092025-11-12T20:53:35.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
167102025-11-12T20:53:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
167112025-11-12T20:53:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
167122025-11-12T20:53:35.412ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
167132025-11-12T20:53:35.412ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
167142025-11-12T20:53:35.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167152025-11-12T20:53:35.412ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
167162025-11-12T20:53:35.412ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
167172025-11-12T20:53:35.412ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 134.266074ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
167182025-11-12T20:53:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
167192025-11-12T20:53:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167202025-11-12T20:53:35.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
167212025-11-12T20:53:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
167222025-11-12T20:53:35.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167232025-11-12T20:53:35.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167242025-11-12T20:53:35.415ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
167252025-11-12T20:53:35.415ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
167262025-11-12T20:53:35.415ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
167272025-11-12T20:53:35.415ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 139.87007ms iteration = 9 status = {"generation":2}
167282025-11-12T20:53:35.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
167292025-11-12T20:53:35.420ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
167302025-11-12T20:53:35.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167312025-11-12T20:53:35.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167322025-11-12T20:53:35.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167332025-11-12T20:53:35.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167342025-11-12T20:53:35.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167352025-11-12T20:53:35.439ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
167362025-11-12T20:53:35.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167372025-11-12T20:53:35.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167382025-11-12T20:53:35.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167392025-11-12T20:53:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167402025-11-12T20:53:35.443ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 163.84776ms iteration = 27 status = {"physical_disks_added":0}
167412025-11-12T20:53:35.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167422025-11-12T20:53:35.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
167432025-11-12T20:53:35.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167442025-11-12T20:53:35.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167452025-11-12T20:53:35.460ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 180.994344ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
167462025-11-12T20:53:35.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167472025-11-12T20:53:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167482025-11-12T20:53:35.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167492025-11-12T20:53:35.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167502025-11-12T20:53:35.484ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = SledAgent step id = Ensure
167512025-11-12T20:53:35.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167522025-11-12T20:53:35.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
167532025-11-12T20:53:35.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167542025-11-12T20:53:35.491ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
167552025-11-12T20:53:35.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167562025-11-12T20:53:35.491ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 215.880207ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
167572025-11-12T20:53:35.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167582025-11-12T20:53:35.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
167592025-11-12T20:53:35.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167602025-11-12T20:53:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167612025-11-12T20:53:35.499ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
167622025-11-12T20:53:35.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
167632025-11-12T20:53:35.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167642025-11-12T20:53:35.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167652025-11-12T20:53:35.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167662025-11-12T20:53:35.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167672025-11-12T20:53:35.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167682025-11-12T20:53:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
167692025-11-12T20:53:35.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167702025-11-12T20:53:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167712025-11-12T20:53:35.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167722025-11-12T20:53:35.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
167732025-11-12T20:53:35.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167742025-11-12T20:53:35.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167752025-11-12T20:53:35.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167762025-11-12T20:53:35.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167772025-11-12T20:53:35.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167782025-11-12T20:53:35.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167792025-11-12T20:53:35.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167802025-11-12T20:53:35.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
167812025-11-12T20:53:35.540ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
167822025-11-12T20:53:35.569ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
167832025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2741b06-0eb4-41fb-a944-b3befb179c16", "content-length": "78", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
167842025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
167852025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
167862025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
167872025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
167882025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
167892025-11-12T20:53:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
167902025-11-12T20:53:35.572ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
167912025-11-12T20:53:35.572ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
167922025-11-12T20:53:35.573ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
167932025-11-12T20:53:35.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
167942025-11-12T20:53:35.576ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
167952025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
167962025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
167972025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
167982025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
167992025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
168002025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
168012025-11-12T20:53:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
168022025-11-12T20:53:35.580ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
168032025-11-12T20:53:35.580ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
168042025-11-12T20:53:35.580ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "07ae2da3-4548-47da-b625-124426fbf78c", "content-length": "81", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168052025-11-12T20:53:35.593ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
168062025-11-12T20:53:35.593ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
168072025-11-12T20:53:35.593ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49f1365c-fbf9-4727-b681-ff1406e113ac", "content-length": "26", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168082025-11-12T20:53:35.593ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
168092025-11-12T20:53:35.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168102025-11-12T20:53:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168112025-11-12T20:53:35.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168122025-11-12T20:53:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168132025-11-12T20:53:35.596ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
168142025-11-12T20:53:35.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168152025-11-12T20:53:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
168162025-11-12T20:53:35.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168172025-11-12T20:53:35.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168182025-11-12T20:53:35.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168192025-11-12T20:53:35.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168202025-11-12T20:53:35.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168212025-11-12T20:53:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
168222025-11-12T20:53:35.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168232025-11-12T20:53:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168242025-11-12T20:53:35.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168252025-11-12T20:53:35.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168262025-11-12T20:53:35.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168272025-11-12T20:53:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
168282025-11-12T20:53:35.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168292025-11-12T20:53:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168302025-11-12T20:53:35.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168312025-11-12T20:53:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168322025-11-12T20:53:35.629ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
168332025-11-12T20:53:35.629ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
168342025-11-12T20:53:35.630ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27eb817-ea6a-4cdc-b89b-1fd3fc89491d", "content-length": "81", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168352025-11-12T20:53:35.632ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
168362025-11-12T20:53:35.632ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
168372025-11-12T20:53:35.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168382025-11-12T20:53:35.633ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fa2da33-41ae-4005-a08a-4e72b3182689", "content-length": "81", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168392025-11-12T20:53:35.635ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 356.435384ms iteration = 17 status = {"disabled":false,"errors":[],"sps":[]}
168402025-11-12T20:53:35.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
168412025-11-12T20:53:35.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168422025-11-12T20:53:35.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168432025-11-12T20:53:35.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168442025-11-12T20:53:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168452025-11-12T20:53:35.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168462025-11-12T20:53:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
168472025-11-12T20:53:35.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168482025-11-12T20:53:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168492025-11-12T20:53:35.661ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
168502025-11-12T20:53:35.661ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1761180c-6c93-4a70-b0ec-e2c2e33650b2", "content-length": "691", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168512025-11-12T20:53:35.661ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
168522025-11-12T20:53:35.661ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
168532025-11-12T20:53:35.661ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
168542025-11-12T20:53:35.663ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 384.204462ms iteration = 17 status = {}
168552025-11-12T20:53:35.687ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
168562025-11-12T20:53:35.690ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
168572025-11-12T20:53:35.690ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 411.995172ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
168582025-11-12T20:53:35.699ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
168592025-11-12T20:53:35.699ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
168602025-11-12T20:53:35.700ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ce1eac9-dacc-4e13-b619-60d2e99c9da5", "content-length": "26", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168612025-11-12T20:53:35.700ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
168622025-11-12T20:53:35.702ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
168632025-11-12T20:53:35.707ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
168642025-11-12T20:53:35.707ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
168652025-11-12T20:53:35.707ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = b5ea5053-760f-491d-b4e6-bdeceab88aef parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
168662025-11-12T20:53:35.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168672025-11-12T20:53:35.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168682025-11-12T20:53:35.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168692025-11-12T20:53:35.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
168702025-11-12T20:53:35.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
168712025-11-12T20:53:35.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
168722025-11-12T20:53:35.716ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
168732025-11-12T20:53:35.717ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "a013d7fc-cc72-4f75-8007-cff776753e82", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168742025-11-12T20:53:35.717ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
168752025-11-12T20:53:35.719ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
168762025-11-12T20:53:35.719ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
168772025-11-12T20:53:35.719ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 440.788477ms iteration = 21 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
168782025-11-12T20:53:35.719ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
168792025-11-12T20:53:35.719ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 427.997546ms iteration = 9 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
168802025-11-12T20:53:35.733ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
168812025-11-12T20:53:35.735ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 459.81538ms iteration = 17 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
168822025-11-12T20:53:35.747ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
168832025-11-12T20:53:35.749ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "f6665d19-684f-43be-b2e9-abb3a6d40761", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
168842025-11-12T20:53:35.751ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = FirewallRules step id = Ensure
168852025-11-12T20:53:35.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
168862025-11-12T20:53:35.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
168872025-11-12T20:53:35.752ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
168882025-11-12T20:53:35.752ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
168892025-11-12T20:53:35.752ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ffaaa68-480f-47e0-b53a-84a0fd1d5f24", "content-length": "372", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
168902025-11-12T20:53:35.752ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
168912025-11-12T20:53:35.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168922025-11-12T20:53:35.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
168932025-11-12T20:53:35.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
168942025-11-12T20:53:35.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
168952025-11-12T20:53:35.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
168962025-11-12T20:53:35.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
168972025-11-12T20:53:35.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
168982025-11-12T20:53:35.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
168992025-11-12T20:53:35.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169002025-11-12T20:53:35.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169012025-11-12T20:53:35.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169022025-11-12T20:53:35.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169032025-11-12T20:53:35.769ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
169042025-11-12T20:53:35.769ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
169052025-11-12T20:53:35.769ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7663ea51-286d-4830-b64e-cdee94dd27c2", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
169062025-11-12T20:53:35.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169072025-11-12T20:53:35.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169082025-11-12T20:53:35.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169092025-11-12T20:53:35.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169102025-11-12T20:53:35.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
169112025-11-12T20:53:35.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169122025-11-12T20:53:35.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169132025-11-12T20:53:35.776ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
169142025-11-12T20:53:35.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169152025-11-12T20:53:35.777ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169162025-11-12T20:53:35.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169172025-11-12T20:53:35.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169182025-11-12T20:53:35.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169192025-11-12T20:53:35.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169202025-11-12T20:53:35.781ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
169212025-11-12T20:53:35.781ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
169222025-11-12T20:53:35.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169232025-11-12T20:53:35.781ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7388720b-d217-4de4-a092-a46f04683494", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
169242025-11-12T20:53:35.784ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 505.228885ms iteration = 25 status = {}
169252025-11-12T20:53:35.784ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Dependency
169262025-11-12T20:53:35.784ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
169272025-11-12T20:53:35.784ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169282025-11-12T20:53:35.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169292025-11-12T20:53:35.786ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169302025-11-12T20:53:35.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
169312025-11-12T20:53:35.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169322025-11-12T20:53:35.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
169332025-11-12T20:53:35.789ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169342025-11-12T20:53:35.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169352025-11-12T20:53:35.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169362025-11-12T20:53:35.791ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169372025-11-12T20:53:35.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169382025-11-12T20:53:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169392025-11-12T20:53:35.794ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
169402025-11-12T20:53:35.794ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169412025-11-12T20:53:35.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169422025-11-12T20:53:35.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169432025-11-12T20:53:35.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169442025-11-12T20:53:35.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169452025-11-12T20:53:35.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169462025-11-12T20:53:35.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169472025-11-12T20:53:35.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169482025-11-12T20:53:35.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169492025-11-12T20:53:35.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169502025-11-12T20:53:35.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169512025-11-12T20:53:35.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169522025-11-12T20:53:35.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169532025-11-12T20:53:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
169542025-11-12T20:53:35.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169552025-11-12T20:53:35.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169562025-11-12T20:53:35.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169572025-11-12T20:53:35.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169582025-11-12T20:53:35.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169592025-11-12T20:53:35.823ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 547.968981ms iteration = 17 status = {}
169602025-11-12T20:53:35.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169612025-11-12T20:53:35.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169622025-11-12T20:53:35.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169632025-11-12T20:53:35.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169642025-11-12T20:53:35.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169652025-11-12T20:53:35.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169662025-11-12T20:53:35.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169672025-11-12T20:53:35.831ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 327492584 } })
169682025-11-12T20:53:35.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169692025-11-12T20:53:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169702025-11-12T20:53:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169712025-11-12T20:53:35.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
169722025-11-12T20:53:35.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169732025-11-12T20:53:35.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169742025-11-12T20:53:35.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169752025-11-12T20:53:35.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169762025-11-12T20:53:35.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169772025-11-12T20:53:35.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169782025-11-12T20:53:35.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169792025-11-12T20:53:35.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
169802025-11-12T20:53:35.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169812025-11-12T20:53:35.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169822025-11-12T20:53:35.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169832025-11-12T20:53:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169842025-11-12T20:53:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169852025-11-12T20:53:35.864ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
169862025-11-12T20:53:35.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169872025-11-12T20:53:35.864ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "0c504db6-50e2-44eb-934a-23ec96dbe74a", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
169882025-11-12T20:53:35.864ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
169892025-11-12T20:53:35.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169902025-11-12T20:53:35.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169912025-11-12T20:53:35.867ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9a9c348a-2258-4106-a0d2-abeeca144a34", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
169922025-11-12T20:53:35.867ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
169932025-11-12T20:53:35.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169942025-11-12T20:53:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169952025-11-12T20:53:35.870ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
169962025-11-12T20:53:35.870ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 594.564236ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
169972025-11-12T20:53:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
169982025-11-12T20:53:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
169992025-11-12T20:53:35.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170002025-11-12T20:53:35.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170012025-11-12T20:53:35.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170022025-11-12T20:53:35.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170032025-11-12T20:53:35.877ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
170042025-11-12T20:53:35.878ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ead0a12c-e8c1-4749-928d-0e578839b16f", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170052025-11-12T20:53:35.878ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170062025-11-12T20:53:35.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170072025-11-12T20:53:35.880ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c9d19a47-23c7-4e88-a29d-d82cc7e3e8c6", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170082025-11-12T20:53:35.880ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170092025-11-12T20:53:35.883ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "2df869bb-29dd-4000-a501-9298fe6f2f8d", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170102025-11-12T20:53:35.883ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170112025-11-12T20:53:35.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
170122025-11-12T20:53:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170132025-11-12T20:53:35.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170142025-11-12T20:53:35.885ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
170152025-11-12T20:53:35.885ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 610.109299ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
170162025-11-12T20:53:35.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170172025-11-12T20:53:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170182025-11-12T20:53:35.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170192025-11-12T20:53:35.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170202025-11-12T20:53:35.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170212025-11-12T20:53:35.895ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
170222025-11-12T20:53:35.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170232025-11-12T20:53:35.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170242025-11-12T20:53:35.904ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
170252025-11-12T20:53:35.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170262025-11-12T20:53:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170272025-11-12T20:53:35.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170282025-11-12T20:53:35.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
170292025-11-12T20:53:35.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170302025-11-12T20:53:35.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170312025-11-12T20:53:35.912ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
170322025-11-12T20:53:35.912ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
170332025-11-12T20:53:35.912ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
170342025-11-12T20:53:35.959ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
170352025-11-12T20:53:35.959ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
170362025-11-12T20:53:35.966ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2488b92b-6d36-4ccc-a7be-21ac7d7a8b6d", "content-length": "372", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
170372025-11-12T20:53:35.966ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
170382025-11-12T20:53:35.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
170392025-11-12T20:53:35.972ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
170402025-11-12T20:53:35.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
170412025-11-12T20:53:35.975ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
170422025-11-12T20:53:35.975ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
170432025-11-12T20:53:35.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
170442025-11-12T20:53:35.981ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
170452025-11-12T20:53:35.981ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
170462025-11-12T20:53:35.986ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "aa2310cb-b470-4845-af3a-2ce6992ed22e", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
170472025-11-12T20:53:35.986ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
170482025-11-12T20:53:35.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
170492025-11-12T20:53:35.989ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
170502025-11-12T20:53:35.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
170512025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
170522025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
170532025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
170542025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
170552025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
170562025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
170572025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
170582025-11-12T20:53:35.991ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
170592025-11-12T20:53:35.992ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
170602025-11-12T20:53:35.992ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
170612025-11-12T20:53:35.992ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
170622025-11-12T20:53:35.992ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
170632025-11-12T20:53:35.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
170642025-11-12T20:53:35.992ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
170652025-11-12T20:53:35.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
170662025-11-12T20:53:35.993ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
170672025-11-12T20:53:35.993ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
170682025-11-12T20:53:35.994ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
170692025-11-12T20:53:35.994ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
170702025-11-12T20:53:35.995ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
170712025-11-12T20:53:35.995ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
170722025-11-12T20:53:35.996ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7bacf5b4-f4b4-4050-87e2-52c460a080cf", "date": "Wed, 12 Nov 2025 20:53:35 GMT"} })
170732025-11-12T20:53:35.997ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 213.178846ms iteration = 26 status = {}
170742025-11-12T20:53:36.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170752025-11-12T20:53:36.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170762025-11-12T20:53:36.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170772025-11-12T20:53:36.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170782025-11-12T20:53:36.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170792025-11-12T20:53:36.039ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
170802025-11-12T20:53:36.039ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
170812025-11-12T20:53:36.040ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ffe33e57-54de-4a83-b2e8-33ebc03ceddc", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
170822025-11-12T20:53:36.042ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
170832025-11-12T20:53:36.042ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = Dns step id = Ensure
170842025-11-12T20:53:36.042ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
170852025-11-12T20:53:36.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170862025-11-12T20:53:36.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170872025-11-12T20:53:36.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170882025-11-12T20:53:36.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170892025-11-12T20:53:36.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170902025-11-12T20:53:36.048ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
170912025-11-12T20:53:36.049ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
170922025-11-12T20:53:36.049ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
170932025-11-12T20:53:36.049ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
170942025-11-12T20:53:36.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170952025-11-12T20:53:36.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
170962025-11-12T20:53:36.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170972025-11-12T20:53:36.054ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
170982025-11-12T20:53:36.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
170992025-11-12T20:53:36.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171002025-11-12T20:53:36.096ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
171012025-11-12T20:53:36.096ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
171022025-11-12T20:53:36.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171032025-11-12T20:53:36.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171042025-11-12T20:53:36.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171052025-11-12T20:53:36.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
171062025-11-12T20:53:36.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171072025-11-12T20:53:36.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171082025-11-12T20:53:36.105ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
171092025-11-12T20:53:36.105ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
171102025-11-12T20:53:36.105ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
171112025-11-12T20:53:36.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171122025-11-12T20:53:36.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171132025-11-12T20:53:36.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171142025-11-12T20:53:36.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171152025-11-12T20:53:36.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171162025-11-12T20:53:36.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
171172025-11-12T20:53:36.140ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0ac446fd-8e54-45a3-bbfa-9695fabf339c", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
171182025-11-12T20:53:36.140ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
171192025-11-12T20:53:36.140ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
171202025-11-12T20:53:36.146ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "72b9ed10-fe25-4f3d-ae76-372bce9179ca", "content-length": "682", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
171212025-11-12T20:53:36.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171222025-11-12T20:53:36.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171232025-11-12T20:53:36.148ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
171242025-11-12T20:53:36.148ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
171252025-11-12T20:53:36.148ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
171262025-11-12T20:53:36.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171272025-11-12T20:53:36.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171282025-11-12T20:53:36.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171292025-11-12T20:53:36.150ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171302025-11-12T20:53:36.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171312025-11-12T20:53:36.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
171322025-11-12T20:53:36.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171332025-11-12T20:53:36.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171342025-11-12T20:53:36.154ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171352025-11-12T20:53:36.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171362025-11-12T20:53:36.156ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
171372025-11-12T20:53:36.156ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
171382025-11-12T20:53:36.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171392025-11-12T20:53:36.156ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171402025-11-12T20:53:36.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171412025-11-12T20:53:36.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171422025-11-12T20:53:36.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171432025-11-12T20:53:36.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
171442025-11-12T20:53:36.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171452025-11-12T20:53:36.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171462025-11-12T20:53:36.166ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
171472025-11-12T20:53:36.166ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
171482025-11-12T20:53:36.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171492025-11-12T20:53:36.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171502025-11-12T20:53:36.167ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 887.561539ms iteration = 17 status = {}
171512025-11-12T20:53:36.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171522025-11-12T20:53:36.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
171532025-11-12T20:53:36.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171542025-11-12T20:53:36.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171552025-11-12T20:53:36.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171562025-11-12T20:53:36.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171572025-11-12T20:53:36.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171582025-11-12T20:53:36.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171592025-11-12T20:53:36.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171602025-11-12T20:53:36.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
171612025-11-12T20:53:36.186ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
171622025-11-12T20:53:36.186ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
171632025-11-12T20:53:36.186ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = OmicronZones step id = Cleanup
171642025-11-12T20:53:36.186ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = OmicronZones step id = Cleanup
171652025-11-12T20:53:36.186ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = PhysicalDisks step id = Cleanup
171662025-11-12T20:53:36.186ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = Clickhouse step id = Ensure
171672025-11-12T20:53:36.186ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = Clickhouse step id = Ensure
171682025-11-12T20:53:36.221ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
171692025-11-12T20:53:36.226ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "58828166-b9d6-4919-a778-35c6d69f2dba", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
171702025-11-12T20:53:36.228ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = SupportBundles step id = Cleanup
171712025-11-12T20:53:36.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171722025-11-12T20:53:36.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171732025-11-12T20:53:36.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171742025-11-12T20:53:36.231ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
171752025-11-12T20:53:36.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171762025-11-12T20:53:36.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171772025-11-12T20:53:36.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171782025-11-12T20:53:36.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171792025-11-12T20:53:36.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171802025-11-12T20:53:36.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
171812025-11-12T20:53:36.278ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = OmicronZones step id = Cleanup
171822025-11-12T20:53:36.278ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
171832025-11-12T20:53:36.278ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
171842025-11-12T20:53:36.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171852025-11-12T20:53:36.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171862025-11-12T20:53:36.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171872025-11-12T20:53:36.282ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
171882025-11-12T20:53:36.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171892025-11-12T20:53:36.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171902025-11-12T20:53:36.317ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
171912025-11-12T20:53:36.317ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
171922025-11-12T20:53:36.317ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = Cockroach step id = Ensure
171932025-11-12T20:53:36.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171942025-11-12T20:53:36.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171952025-11-12T20:53:36.337ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
171962025-11-12T20:53:36.337ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 337b3d51-7ce1-4bea-a54d-d520154e454e step component = MgsUpdates step id = Ensure
171972025-11-12T20:53:36.337ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Dependency
171982025-11-12T20:53:36.337ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Dependency
171992025-11-12T20:53:36.337ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
172002025-11-12T20:53:36.337ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172012025-11-12T20:53:36.338ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.062447365s iteration = 10 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3269,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":94780183,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":94780133,"secs":0}},"event_index":1,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":94794003,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":34896083,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":34896033,"secs":0}},"event_index":2,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":129710466,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":79619890,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":79619840,"secs":0}},"event_index":3,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":209345917,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":266473436,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":266473396,"secs":0}},"event_index":4,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":475841484,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":290544859,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":290544809,"secs":0}},"event_index":5,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":766399803,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":144529441,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":144529411,"secs":0}},"event_index":6,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":910944254,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4600,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4560,"secs":0}},"event_index":7,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":910963984,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2230,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2200,"secs":0}},"event_index":8,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":910968884,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4040,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4000,"secs":0}},"event_index":9,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":910975094,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1960,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1931,"secs":0}},"event_index":10,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":910978855,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41655067,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":41655027,"secs":0}},"event_index":11,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":952635482,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50503287,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50503247,"secs":0}},"event_index":12,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3153980,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38998560,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":38998509,"secs":0}},"event_index":13,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":42169279,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19871814,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19871704,"secs":0}},"event_index":14,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":62049274,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4340,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4300,"secs":0}},"event_index":15,"execution_id":"cb1f2d54-d046-41c8-ba21-3785841a16fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":62061534,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
172022025-11-12T20:53:36.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
172032025-11-12T20:53:36.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172042025-11-12T20:53:36.339ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172052025-11-12T20:53:36.339ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172062025-11-12T20:53:36.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
172072025-11-12T20:53:36.342ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
172082025-11-12T20:53:36.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
172092025-11-12T20:53:36.342ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
172102025-11-12T20:53:36.342ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172112025-11-12T20:53:36.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172122025-11-12T20:53:36.385ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
172132025-11-12T20:53:36.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
172142025-11-12T20:53:36.387ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172152025-11-12T20:53:36.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
172162025-11-12T20:53:36.389ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
172172025-11-12T20:53:36.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
172182025-11-12T20:53:36.391ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
172192025-11-12T20:53:36.391ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 53.889624ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
172202025-11-12T20:53:36.416ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(5b8b6c4c-fea2-4427-b744-2d0ccac60ceb, 0), (fefb374a-1971-42d6-a30a-ae5b0632e967, 0), (f1d519c6-906b-463e-a78e-1e59b98d7eac, 0), (49918508-77d9-410a-ad8e-292b92856858, 0)] collection_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb
172212025-11-12T20:53:36.416ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172222025-11-12T20:53:36.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172232025-11-12T20:53:36.585ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 5b8b6c4c-fea2-4427-b744-2d0ccac60ceb file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
172242025-11-12T20:53:36.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172252025-11-12T20:53:36.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172262025-11-12T20:53:36.589ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(fefb374a-1971-42d6-a30a-ae5b0632e967, 0), (f1d519c6-906b-463e-a78e-1e59b98d7eac, 0), (49918508-77d9-410a-ad8e-292b92856858, 0)]
172272025-11-12T20:53:36.589ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1412, tv_nsec: 85794228 } })
172282025-11-12T20:53:36.622ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1412, tv_nsec: 118666519 } })
172292025-11-12T20:53:36.623ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
172302025-11-12T20:53:36.623ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
172312025-11-12T20:53:36.623ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
172322025-11-12T20:53:36.624ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "af17fbc3-853a-497d-84e1-0dfb2d72cb93", "content-length": "476", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172332025-11-12T20:53:36.624ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
172342025-11-12T20:53:36.624ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec4b7cb5-6bca-4035-b76b-d6b5da797fd7", "content-length": "734", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172352025-11-12T20:53:36.624ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
172362025-11-12T20:53:36.625ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "927db9d5-dd47-4da3-9576-1b59c4f7f753", "content-length": "111", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172372025-11-12T20:53:36.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
172382025-11-12T20:53:36.625ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4465817-ce62-443d-b3c6-102f3b0fe92f", "content-length": "111", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172392025-11-12T20:53:36.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
172402025-11-12T20:53:36.625ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33312497-6b77-4b4e-b0f9-a047f29d9360", "content-length": "163", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172412025-11-12T20:53:36.625ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
172422025-11-12T20:53:36.626ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f40cb268-775b-4a43-a1ac-48e8658d38a8", "content-length": "163", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172432025-11-12T20:53:36.626ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
172442025-11-12T20:53:36.626ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7894a9aa-3eae-4f33-9ae3-3d477913ca32", "content-length": "166", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172452025-11-12T20:53:36.626ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
172462025-11-12T20:53:36.626ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35eeb550-91b3-4e39-989a-6285fb2c7cd7", "content-length": "166", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172472025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
172482025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86ecb695-c84d-41f0-848b-03bf8eac3eba", "content-length": "702", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172492025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
172502025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "199f0cba-12c8-4436-974c-f7049d870221", "content-length": "727", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172512025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
172522025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "245c857b-6725-454d-b34e-643fac77e874", "content-length": "729", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172532025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
172542025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62dd0cdb-0a28-423f-a63a-5d08b384b70d", "content-length": "728", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172552025-11-12T20:53:36.627ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
172562025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38105fc0-1782-404e-b9cb-9c13af67cb8c", "content-length": "722", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172572025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
172582025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca362955-476b-4364-889d-6d1460f58d85", "content-length": "10", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172592025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
172602025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87bb0c8a-4fe2-4495-a4cd-94ccd8c01ebd", "content-length": "144", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172612025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
172622025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c6e1579-edd4-4994-a7d3-d22a1030b1a6", "content-length": "147", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172632025-11-12T20:53:36.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
172642025-11-12T20:53:36.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be926e87-2e90-4948-8747-d7b7d55cb75b", "content-length": "109", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172652025-11-12T20:53:36.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
172662025-11-12T20:53:36.629ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "508dddf8-e3ff-4a01-bb95-333c8f04aeca", "content-length": "109", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172672025-11-12T20:53:36.629ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
172682025-11-12T20:53:36.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2d364fe-b456-4ac2-9fdb-dfa64749ca3d", "content-length": "163", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172692025-11-12T20:53:36.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
172702025-11-12T20:53:36.630ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3332f6c4-184f-43f4-a07b-146779bc0370", "content-length": "163", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172712025-11-12T20:53:36.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
172722025-11-12T20:53:36.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dbdfa26-dbad-43e9-a86a-761f2305ee6a", "content-length": "166", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172732025-11-12T20:53:36.631ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
172742025-11-12T20:53:36.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "75ceca43-f12e-41da-8076-60019f2c7f3b", "content-length": "166", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172752025-11-12T20:53:36.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
172762025-11-12T20:53:36.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18690c04-284c-46b5-8a70-321fc81fb7a9", "content-length": "702", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172772025-11-12T20:53:36.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
172782025-11-12T20:53:36.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b352a8e5-abda-4ab4-9c3a-95b99968da66", "content-length": "727", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172792025-11-12T20:53:36.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
172802025-11-12T20:53:36.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23072ffe-aaaa-4766-83c0-10df61555484", "content-length": "729", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172812025-11-12T20:53:36.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
172822025-11-12T20:53:36.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa48dd14-b89f-4c53-8b2a-39e4454a886d", "content-length": "728", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172832025-11-12T20:53:36.633ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172842025-11-12T20:53:36.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172852025-11-12T20:53:36.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172862025-11-12T20:53:36.639ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
172872025-11-12T20:53:36.639ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172882025-11-12T20:53:36.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172892025-11-12T20:53:36.642ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172902025-11-12T20:53:36.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
172912025-11-12T20:53:36.645ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172922025-11-12T20:53:36.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172932025-11-12T20:53:36.680ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
172942025-11-12T20:53:36.680ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
172952025-11-12T20:53:36.769ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "386565c0-1b37-4bce-8764-ea55780f43b7", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
172962025-11-12T20:53:36.771ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
172972025-11-12T20:53:36.771ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
172982025-11-12T20:53:36.771ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
172992025-11-12T20:53:36.771ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
173002025-11-12T20:53:36.771ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
173012025-11-12T20:53:36.771ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
173022025-11-12T20:53:36.771ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
173032025-11-12T20:53:36.771ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
173042025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "29351a9a-b28a-4eb5-8397-caee650afb1a", "content-length": "64", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173052025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
173062025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8795672a-1e07-4112-9e24-4d2ec1c826cb", "content-length": "64", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173072025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5be1aa40-b949-463f-8536-c754117155e6", "content-length": "64", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173082025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a585ee6-5794-456e-a221-b5c8dd5fbbf0", "content-length": "64", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173092025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
173102025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e6625be-1c3d-41c0-8ce0-dee4dcc5300e", "content-length": "64", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173112025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
173122025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
173132025-11-12T20:53:36.772ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
173142025-11-12T20:53:36.815ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cfe5549-471a-4a6a-9b7d-0d97d7dc6ddc", "content-length": "400659", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173152025-11-12T20:53:36.815ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee5ac030-8fac-4a11-80c4-d724dd47c6fe", "content-length": "401054", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173162025-11-12T20:53:36.815ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "99034e45-bf71-4b16-9344-85ec2740998b", "content-length": "398755", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173172025-11-12T20:53:36.818ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a9207fe-459e-4669-a81f-3b0c61bb42a5", "content-length": "428720", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173182025-11-12T20:53:36.819ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6797d414-c097-4c5f-81eb-ededac5936b5", "content-length": "402774", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173192025-11-12T20:53:36.821ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
173202025-11-12T20:53:36.821ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
173212025-11-12T20:53:36.821ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
173222025-11-12T20:53:36.821ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
173232025-11-12T20:53:36.821ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
173242025-11-12T20:53:36.852ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
173252025-11-12T20:53:36.852ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
173262025-11-12T20:53:36.854ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "64061356-5993-465e-b47a-07ec963da7fb", "content-length": "127", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173272025-11-12T20:53:36.856ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
173282025-11-12T20:53:36.949ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
173292025-11-12T20:53:36.949ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
173302025-11-12T20:53:36.949ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3f5114d-c47c-4317-952e-b0bbfed9dd69", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173312025-11-12T20:53:36.950ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
173322025-11-12T20:53:36.951ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
173332025-11-12T20:53:36.951ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
173342025-11-12T20:53:36.952ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f68e6618-6b2c-4c64-bbeb-f93d1b49c8bc", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173352025-11-12T20:53:36.952ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
173362025-11-12T20:53:36.953ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
173372025-11-12T20:53:36.953ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
173382025-11-12T20:53:36.954ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d9939d6-80d3-4205-a74c-c14b1887ac8a", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:53:36 GMT"} })
173392025-11-12T20:53:36.954ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
173402025-11-12T20:53:36.955ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
173412025-11-12T20:53:36.955ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
173422025-11-12T20:53:36.957ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
173432025-11-12T20:53:36.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
173442025-11-12T20:53:36.959ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173452025-11-12T20:53:36.962ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
173462025-11-12T20:53:36.962ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
173472025-11-12T20:53:36.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
173482025-11-12T20:53:37.168ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = e3133932-a041-4d2e-804a-0de86a112082 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
173492025-11-12T20:53:37.169ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = e3133932-a041-4d2e-804a-0de86a112082 time_started = 2025-11-12 20:53:36.623549 UTC
173502025-11-12T20:53:37.169ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 43 reason = Dependency
173512025-11-12T20:53:37.169ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
173522025-11-12T20:53:37.169ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 831.282472ms iteration = 10 status = {"collection_id":"e3133932-a041-4d2e-804a-0de86a112082","time_done":"2025-11-12 20:53:36.955653 UTC","time_started":"2025-11-12 20:53:36.623549 UTC"}
173532025-11-12T20:53:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
173542025-11-12T20:53:37.171ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173552025-11-12T20:53:37.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
173562025-11-12T20:53:37.175ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
173572025-11-12T20:53:37.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
173582025-11-12T20:53:37.177ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
173592025-11-12T20:53:37.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
173602025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Dependency
173612025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 54.043718ms iteration = 43 status = {"Loaded":{"collection_id":"e3133932-a041-4d2e-804a-0de86a112082","time_loaded":"2025-11-12T20:53:37.169003676Z","time_started":"2025-11-12T20:53:36.623549Z"}}
173622025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 22 reason = Dependency
173632025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 13 reason = Dependency
173642025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
173652025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173662025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173672025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173682025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
173692025-11-12T20:53:37.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173702025-11-12T20:53:37.224ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173712025-11-12T20:53:37.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173722025-11-12T20:53:37.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173732025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
173742025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173752025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
173762025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
173772025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
173782025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173792025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173802025-11-12T20:53:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
173812025-11-12T20:53:37.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173822025-11-12T20:53:37.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173832025-11-12T20:53:37.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173842025-11-12T20:53:37.232ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
173852025-11-12T20:53:37.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
173862025-11-12T20:53:37.233ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
173872025-11-12T20:53:37.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173882025-11-12T20:53:37.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173892025-11-12T20:53:37.234ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173902025-11-12T20:53:37.235ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
173912025-11-12T20:53:37.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173922025-11-12T20:53:37.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173932025-11-12T20:53:37.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173942025-11-12T20:53:37.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173952025-11-12T20:53:37.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
173962025-11-12T20:53:37.236ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
173972025-11-12T20:53:37.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
173982025-11-12T20:53:37.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173992025-11-12T20:53:37.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174002025-11-12T20:53:37.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174012025-11-12T20:53:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174022025-11-12T20:53:37.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
174032025-11-12T20:53:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174042025-11-12T20:53:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174052025-11-12T20:53:37.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174062025-11-12T20:53:37.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174072025-11-12T20:53:37.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174082025-11-12T20:53:37.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174092025-11-12T20:53:37.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174102025-11-12T20:53:37.243ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
174112025-11-12T20:53:37.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174122025-11-12T20:53:37.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174132025-11-12T20:53:37.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174142025-11-12T20:53:37.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174152025-11-12T20:53:37.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174162025-11-12T20:53:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174172025-11-12T20:53:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174182025-11-12T20:53:37.247ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
174192025-11-12T20:53:37.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174202025-11-12T20:53:37.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174212025-11-12T20:53:37.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174222025-11-12T20:53:37.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174232025-11-12T20:53:37.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174242025-11-12T20:53:37.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174252025-11-12T20:53:37.251ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174262025-11-12T20:53:37.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174272025-11-12T20:53:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174282025-11-12T20:53:37.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174292025-11-12T20:53:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174302025-11-12T20:53:37.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174312025-11-12T20:53:37.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174322025-11-12T20:53:37.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174332025-11-12T20:53:37.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174342025-11-12T20:53:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174352025-11-12T20:53:37.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174362025-11-12T20:53:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174372025-11-12T20:53:37.260ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
174382025-11-12T20:53:37.260ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 37.067639ms iteration = 13 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"e3133932-a041-4d2e-804a-0de86a112082","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
174392025-11-12T20:53:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174402025-11-12T20:53:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
174412025-11-12T20:53:37.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174422025-11-12T20:53:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174432025-11-12T20:53:37.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174442025-11-12T20:53:37.267ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174452025-11-12T20:53:37.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174462025-11-12T20:53:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174472025-11-12T20:53:37.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174482025-11-12T20:53:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174492025-11-12T20:53:37.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174502025-11-12T20:53:37.274ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
174512025-11-12T20:53:37.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174522025-11-12T20:53:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174532025-11-12T20:53:37.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174542025-11-12T20:53:37.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174552025-11-12T20:53:37.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174562025-11-12T20:53:37.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
174572025-11-12T20:53:37.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174582025-11-12T20:53:37.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174592025-11-12T20:53:37.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174602025-11-12T20:53:37.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174612025-11-12T20:53:37.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174622025-11-12T20:53:37.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
174632025-11-12T20:53:37.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174642025-11-12T20:53:37.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174652025-11-12T20:53:37.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174662025-11-12T20:53:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174672025-11-12T20:53:37.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174682025-11-12T20:53:37.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174692025-11-12T20:53:37.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174702025-11-12T20:53:37.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174712025-11-12T20:53:37.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174722025-11-12T20:53:37.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174732025-11-12T20:53:37.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174742025-11-12T20:53:37.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174752025-11-12T20:53:37.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174762025-11-12T20:53:37.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174772025-11-12T20:53:37.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174782025-11-12T20:53:37.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174792025-11-12T20:53:37.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174802025-11-12T20:53:37.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174812025-11-12T20:53:37.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174822025-11-12T20:53:37.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174832025-11-12T20:53:37.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174842025-11-12T20:53:37.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174852025-11-12T20:53:37.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174862025-11-12T20:53:37.340ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174872025-11-12T20:53:37.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174882025-11-12T20:53:37.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174892025-11-12T20:53:37.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174902025-11-12T20:53:37.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174912025-11-12T20:53:37.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174922025-11-12T20:53:37.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174932025-11-12T20:53:37.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174942025-11-12T20:53:37.349ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
174952025-11-12T20:53:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174962025-11-12T20:53:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174972025-11-12T20:53:37.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174982025-11-12T20:53:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174992025-11-12T20:53:37.353ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 130.161119ms iteration = 28 status = {"physical_disks_added":0}
175002025-11-12T20:53:37.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175012025-11-12T20:53:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
175022025-11-12T20:53:37.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175032025-11-12T20:53:37.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175042025-11-12T20:53:37.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175052025-11-12T20:53:37.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175062025-11-12T20:53:37.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175072025-11-12T20:53:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
175082025-11-12T20:53:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175092025-11-12T20:53:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175102025-11-12T20:53:37.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175112025-11-12T20:53:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175122025-11-12T20:53:37.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175132025-11-12T20:53:37.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
175142025-11-12T20:53:37.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175152025-11-12T20:53:37.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175162025-11-12T20:53:37.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175172025-11-12T20:53:37.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175182025-11-12T20:53:37.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175192025-11-12T20:53:37.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
175202025-11-12T20:53:37.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175212025-11-12T20:53:37.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175222025-11-12T20:53:37.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175232025-11-12T20:53:37.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175242025-11-12T20:53:37.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175252025-11-12T20:53:37.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
175262025-11-12T20:53:37.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175272025-11-12T20:53:37.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175282025-11-12T20:53:37.425ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
175292025-11-12T20:53:37.425ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
175302025-11-12T20:53:37.425ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 904c3300-5068-4afe-bf30-4d2aa1c40666 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
175312025-11-12T20:53:37.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175322025-11-12T20:53:37.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175332025-11-12T20:53:37.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175342025-11-12T20:53:37.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
175352025-11-12T20:53:37.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175362025-11-12T20:53:37.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175372025-11-12T20:53:37.434ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
175382025-11-12T20:53:37.434ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
175392025-11-12T20:53:37.434ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 211.203866ms iteration = 22 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
175402025-11-12T20:53:37.939ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 396 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 56a7faef-63a6-41c1-8a8a-cbb1428dbf3d response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
175412025-11-12T20:53:40.279ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 98 reason = Timeout
175422025-11-12T20:53:40.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175432025-11-12T20:53:40.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175442025-11-12T20:53:40.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175452025-11-12T20:53:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
175462025-11-12T20:53:40.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175472025-11-12T20:53:40.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175482025-11-12T20:53:40.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
175492025-11-12T20:53:40.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.877918ms iteration = 98 status = {"config_updated":false}
175502025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
175512025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
175522025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
175532025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 99 reason = Timeout
175542025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
175552025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175562025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
175572025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
175582025-11-12T20:53:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175592025-11-12T20:53:45.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
175602025-11-12T20:53:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175612025-11-12T20:53:45.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175622025-11-12T20:53:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
175632025-11-12T20:53:45.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.669185ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
175642025-11-12T20:53:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
175652025-11-12T20:53:45.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
175662025-11-12T20:53:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
175672025-11-12T20:53:45.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175682025-11-12T20:53:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
175692025-11-12T20:53:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175702025-11-12T20:53:45.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
175712025-11-12T20:53:45.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.900805ms iteration = 99 status = {"config_updated":false}
175722025-11-12T20:53:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
175732025-11-12T20:53:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
175742025-11-12T20:53:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175752025-11-12T20:53:45.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
175762025-11-12T20:53:45.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
175772025-11-12T20:53:45.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
175782025-11-12T20:53:45.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
175792025-11-12T20:53:45.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.2664ms iteration = 50 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
175802025-11-12T20:53:47.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 368 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = b147d27e-d3b6-45c1-80f5-286defbc0452 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
175812025-11-12T20:53:50.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 44 reason = Timeout
175822025-11-12T20:53:50.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
175832025-11-12T20:53:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
175842025-11-12T20:53:50.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 100 reason = Timeout
175852025-11-12T20:53:50.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175862025-11-12T20:53:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175872025-11-12T20:53:50.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175882025-11-12T20:53:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175892025-11-12T20:53:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
175902025-11-12T20:53:50.283ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
175912025-11-12T20:53:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
175922025-11-12T20:53:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
175932025-11-12T20:53:50.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175942025-11-12T20:53:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175952025-11-12T20:53:50.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 34 reason = Timeout
175962025-11-12T20:53:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
175972025-11-12T20:53:50.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e3133932-a041-4d2e-804a-0de86a112082 original_time_started = 2025-11-12 20:53:36.623549 UTC
175982025-11-12T20:53:50.285ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.900365ms iteration = 44 status = {"Loaded":{"collection_id":"e3133932-a041-4d2e-804a-0de86a112082","time_loaded":"2025-11-12T20:53:50.277647498Z","time_started":"2025-11-12T20:53:36.623549Z"}}
175992025-11-12T20:53:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
176002025-11-12T20:53:50.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
176012025-11-12T20:53:50.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.824853ms iteration = 100 status = {"config_updated":false}
176022025-11-12T20:53:50.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176032025-11-12T20:53:50.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
176042025-11-12T20:53:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
176052025-11-12T20:53:50.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
176062025-11-12T20:53:50.290ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.580584ms iteration = 34 status = "NoSitrep"
176072025-11-12T20:53:50.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 34 reason = Timeout
176082025-11-12T20:53:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
176092025-11-12T20:53:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
176102025-11-12T20:53:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
176112025-11-12T20:53:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
176122025-11-12T20:53:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176132025-11-12T20:53:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
176142025-11-12T20:53:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
176152025-11-12T20:53:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
176162025-11-12T20:53:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
176172025-11-12T20:53:50.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
176182025-11-12T20:53:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176192025-11-12T20:53:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
176202025-11-12T20:53:50.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
176212025-11-12T20:53:50.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.764022ms iteration = 34 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
176222025-11-12T20:53:55.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
176232025-11-12T20:53:55.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
176242025-11-12T20:53:55.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
176252025-11-12T20:53:55.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 101 reason = Timeout
176262025-11-12T20:53:55.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176272025-11-12T20:53:55.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
176282025-11-12T20:53:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
176292025-11-12T20:53:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176302025-11-12T20:53:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
176312025-11-12T20:53:55.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176322025-11-12T20:53:55.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
176332025-11-12T20:53:55.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176342025-11-12T20:53:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
176352025-11-12T20:53:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
176362025-11-12T20:53:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176372025-11-12T20:53:55.285ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 6.970261ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
176382025-11-12T20:53:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176392025-11-12T20:53:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
176402025-11-12T20:53:55.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
176412025-11-12T20:53:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
176422025-11-12T20:53:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176432025-11-12T20:53:55.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176442025-11-12T20:53:55.288ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
176452025-11-12T20:53:55.288ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.972608ms iteration = 101 status = {"config_updated":false}
176462025-11-12T20:53:55.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176472025-11-12T20:53:55.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
176482025-11-12T20:53:55.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176492025-11-12T20:53:55.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176502025-11-12T20:53:55.313ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
176512025-11-12T20:53:55.313ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.583465ms iteration = 51 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
176522025-11-12T20:53:57.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 375 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 08d6b2e2-4e67-4ef6-928b-0785bb3915d0 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
176532025-11-12T20:54:00.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 102 reason = Timeout
176542025-11-12T20:54:00.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
176552025-11-12T20:54:00.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
176562025-11-12T20:54:00.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176572025-11-12T20:54:00.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
176582025-11-12T20:54:00.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
176592025-11-12T20:54:00.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
176602025-11-12T20:54:00.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
176612025-11-12T20:54:00.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.062833ms iteration = 102 status = {"config_updated":false}
176622025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
176632025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
176642025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
176652025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
176662025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
176672025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
176682025-11-12T20:54:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
176692025-11-12T20:54:05.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 45 reason = Timeout
176702025-11-12T20:54:05.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
176712025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
176722025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
176732025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
176742025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
176752025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
176762025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
176772025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
176782025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
176792025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
176802025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
176812025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
176822025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
176832025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
176842025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
176852025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
176862025-11-12T20:54:05.278ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
176872025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
176882025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
176892025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
176902025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
176912025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
176922025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
176932025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
176942025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
176952025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
176962025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
176972025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
176982025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
176992025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
177002025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
177012025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
177022025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
177032025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
177042025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
177052025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
177062025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
177072025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 103 reason = Timeout
177082025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
177092025-11-12T20:54:05.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1440, tv_nsec: 774511582 } })
177102025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
177112025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
177122025-11-12T20:54:05.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1440, tv_nsec: 774724828 } })
177132025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
177142025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
177152025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
177162025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
177172025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
177182025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
177192025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
177202025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
177212025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
177222025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
177232025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
177242025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
177252025-11-12T20:54:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
177262025-11-12T20:54:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
177272025-11-12T20:54:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
177282025-11-12T20:54:05.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1440, tv_nsec: 775595640 } })
177292025-11-12T20:54:05.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
177302025-11-12T20:54:05.280ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.978599ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
177312025-11-12T20:54:05.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177322025-11-12T20:54:05.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.988378ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
177332025-11-12T20:54:05.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177342025-11-12T20:54:05.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177352025-11-12T20:54:05.283ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
177362025-11-12T20:54:05.283ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
177372025-11-12T20:54:05.283ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
177382025-11-12T20:54:05.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
177392025-11-12T20:54:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
177402025-11-12T20:54:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
177412025-11-12T20:54:05.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 35 reason = Timeout
177422025-11-12T20:54:05.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
177432025-11-12T20:54:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
177442025-11-12T20:54:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
177452025-11-12T20:54:05.285ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
177462025-11-12T20:54:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
177472025-11-12T20:54:05.287ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.505386ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
177482025-11-12T20:54:05.287ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
177492025-11-12T20:54:05.287ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
177502025-11-12T20:54:05.287ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
177512025-11-12T20:54:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
177522025-11-12T20:54:05.287ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
177532025-11-12T20:54:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
177542025-11-12T20:54:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
177552025-11-12T20:54:05.288ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
177562025-11-12T20:54:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
177572025-11-12T20:54:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
177582025-11-12T20:54:05.289ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e3133932-a041-4d2e-804a-0de86a112082 original_time_started = 2025-11-12 20:53:36.623549 UTC
177592025-11-12T20:54:05.289ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.424522ms iteration = 45 status = {"Loaded":{"collection_id":"e3133932-a041-4d2e-804a-0de86a112082","time_loaded":"2025-11-12T20:54:05.277416148Z","time_started":"2025-11-12T20:53:36.623549Z"}}
177602025-11-12T20:54:05.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 35 reason = Timeout
177612025-11-12T20:54:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
177622025-11-12T20:54:05.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177632025-11-12T20:54:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
177642025-11-12T20:54:05.296ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
177652025-11-12T20:54:05.296ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 17.729229ms iteration = 103 status = {"config_updated":false}
177662025-11-12T20:54:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
177672025-11-12T20:54:05.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
177682025-11-12T20:54:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
177692025-11-12T20:54:05.300ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.752369ms iteration = 18 status = 0
177702025-11-12T20:54:05.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
177712025-11-12T20:54:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
177722025-11-12T20:54:05.303ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
177732025-11-12T20:54:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
177742025-11-12T20:54:05.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177752025-11-12T20:54:05.307ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177762025-11-12T20:54:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
177772025-11-12T20:54:05.308ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
177782025-11-12T20:54:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
177792025-11-12T20:54:05.310ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
177802025-11-12T20:54:05.315ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
177812025-11-12T20:54:05.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
177822025-11-12T20:54:05.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
177832025-11-12T20:54:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
177842025-11-12T20:54:05.333ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 55.144757ms iteration = 29 status = {"physical_disks_added":0}
177852025-11-12T20:54:05.339ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177862025-11-12T20:54:05.342ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
177872025-11-12T20:54:05.343ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
177882025-11-12T20:54:05.343ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 51.533733ms iteration = 35 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
177892025-11-12T20:54:05.361ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177902025-11-12T20:54:05.362ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
177912025-11-12T20:54:05.364ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177922025-11-12T20:54:05.364ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 89.515357ms iteration = 18 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
177932025-11-12T20:54:05.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177942025-11-12T20:54:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
177952025-11-12T20:54:05.366ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
177962025-11-12T20:54:05.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
177972025-11-12T20:54:05.368ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
177982025-11-12T20:54:05.368ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
177992025-11-12T20:54:05.368ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 90.630556ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
178002025-11-12T20:54:05.368ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
178012025-11-12T20:54:05.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
178022025-11-12T20:54:05.369ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178032025-11-12T20:54:05.369ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
178042025-11-12T20:54:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
178052025-11-12T20:54:05.369ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1440, tv_nsec: 865631190 } })
178062025-11-12T20:54:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178072025-11-12T20:54:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
178082025-11-12T20:54:05.370ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
178092025-11-12T20:54:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
178102025-11-12T20:54:05.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
178112025-11-12T20:54:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
178122025-11-12T20:54:05.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178132025-11-12T20:54:05.371ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
178142025-11-12T20:54:05.371ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 93.829139ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
178152025-11-12T20:54:05.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
178162025-11-12T20:54:05.372ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178172025-11-12T20:54:05.373ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
178182025-11-12T20:54:05.373ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
178192025-11-12T20:54:05.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
178202025-11-12T20:54:05.373ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "41dd63da-f0c7-4959-a0c2-a37e8acfd7e7", "content-length": "26", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
178212025-11-12T20:54:05.373ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
178222025-11-12T20:54:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
178232025-11-12T20:54:05.374ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 95.8333ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
178242025-11-12T20:54:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
178252025-11-12T20:54:05.374ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
178262025-11-12T20:54:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
178272025-11-12T20:54:05.375ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178282025-11-12T20:54:05.375ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 96.99069ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
178292025-11-12T20:54:05.375ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
178302025-11-12T20:54:05.375ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
178312025-11-12T20:54:05.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178322025-11-12T20:54:05.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
178332025-11-12T20:54:05.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
178342025-11-12T20:54:05.376ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178352025-11-12T20:54:05.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178362025-11-12T20:54:05.377ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 99.355181ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
178372025-11-12T20:54:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
178382025-11-12T20:54:05.379ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
178392025-11-12T20:54:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
178402025-11-12T20:54:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
178412025-11-12T20:54:05.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
178422025-11-12T20:54:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
178432025-11-12T20:54:05.385ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178442025-11-12T20:54:05.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
178452025-11-12T20:54:05.388ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
178462025-11-12T20:54:05.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
178472025-11-12T20:54:05.391ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
178482025-11-12T20:54:05.391ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 113.493027ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
178492025-11-12T20:54:05.393ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 108.454677ms iteration = 35 status = "NoSitrep"
178502025-11-12T20:54:05.414ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
178512025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd0aea0a-393e-4755-b90e-9a87607953bc", "content-length": "78", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
178522025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
178532025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
178542025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
178552025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
178562025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
178572025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
178582025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
178592025-11-12T20:54:05.415ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 137.397785ms iteration = 52 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
178602025-11-12T20:54:05.417ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
178612025-11-12T20:54:05.418ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
178622025-11-12T20:54:05.421ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
178632025-11-12T20:54:05.421ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
178642025-11-12T20:54:05.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
178652025-11-12T20:54:05.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
178662025-11-12T20:54:05.423ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
178672025-11-12T20:54:05.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
178682025-11-12T20:54:05.434ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
178692025-11-12T20:54:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
178702025-11-12T20:54:05.435ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
178712025-11-12T20:54:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
178722025-11-12T20:54:05.437ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178732025-11-12T20:54:05.439ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
178742025-11-12T20:54:05.439ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
178752025-11-12T20:54:05.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
178762025-11-12T20:54:05.449ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
178772025-11-12T20:54:05.449ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
178782025-11-12T20:54:05.450ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "77b9e498-8f78-46dd-a1d0-c523a11c8389", "content-length": "26", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
178792025-11-12T20:54:05.450ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
178802025-11-12T20:54:05.452ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
178812025-11-12T20:54:05.471ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 195.964551ms iteration = 18 status = {}
178822025-11-12T20:54:05.479ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
178832025-11-12T20:54:05.479ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 201.71493ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
178842025-11-12T20:54:05.480ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
178852025-11-12T20:54:05.480ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
178862025-11-12T20:54:05.480ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ce8b4bb-9cd5-4f78-81bf-2a40e85e5d21", "content-length": "81", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
178872025-11-12T20:54:05.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 218.866364ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
178882025-11-12T20:54:05.503ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1440, tv_nsec: 999775512 } })
178892025-11-12T20:54:05.508ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
178902025-11-12T20:54:05.508ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
178912025-11-12T20:54:05.508ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf18062c-4017-4221-8a4f-d78d22b204dc", "content-length": "81", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
178922025-11-12T20:54:05.509ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
178932025-11-12T20:54:05.513ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
178942025-11-12T20:54:05.513ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
178952025-11-12T20:54:05.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
178962025-11-12T20:54:05.515ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
178972025-11-12T20:54:05.515ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
178982025-11-12T20:54:05.515ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "7697b509-4919-45b7-b34d-8a95f53bae1d", "content-length": "81", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
178992025-11-12T20:54:05.517ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 239.19778ms iteration = 18 status = {"disabled":false,"errors":[],"sps":[]}
179002025-11-12T20:54:05.524ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179012025-11-12T20:54:05.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179022025-11-12T20:54:05.551ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
179032025-11-12T20:54:05.551ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
179042025-11-12T20:54:05.551ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a851da80-8a4b-4b60-882f-e7b3ca9167ef", "content-length": "372", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
179052025-11-12T20:54:05.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
179062025-11-12T20:54:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
179072025-11-12T20:54:05.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179082025-11-12T20:54:05.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179092025-11-12T20:54:05.595ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
179102025-11-12T20:54:05.595ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179112025-11-12T20:54:05.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179122025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
179132025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
179142025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
179152025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
179162025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
179172025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
179182025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
179192025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
179202025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
179212025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
179222025-11-12T20:54:05.609ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
179232025-11-12T20:54:05.610ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179242025-11-12T20:54:05.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179252025-11-12T20:54:05.612ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
179262025-11-12T20:54:05.612ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
179272025-11-12T20:54:05.613ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f63dea1f-a1f8-4250-b43a-bc9fb43158d9", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
179282025-11-12T20:54:05.613ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
179292025-11-12T20:54:05.613ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
179302025-11-12T20:54:05.613ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
179312025-11-12T20:54:05.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
179322025-11-12T20:54:05.615ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
179332025-11-12T20:54:05.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
179342025-11-12T20:54:05.619ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
179352025-11-12T20:54:05.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
179362025-11-12T20:54:05.622ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
179372025-11-12T20:54:05.622ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
179382025-11-12T20:54:05.623ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fc04a438-eb1c-4b63-ace7-929448480437", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
179392025-11-12T20:54:05.624ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 346.300962ms iteration = 27 status = {}
179402025-11-12T20:54:05.654ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179412025-11-12T20:54:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179422025-11-12T20:54:05.656ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179432025-11-12T20:54:05.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
179442025-11-12T20:54:05.658ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179452025-11-12T20:54:05.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179462025-11-12T20:54:05.660ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179472025-11-12T20:54:05.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
179482025-11-12T20:54:05.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179492025-11-12T20:54:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179502025-11-12T20:54:05.695ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
179512025-11-12T20:54:05.696ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0a6af5f-5391-4b57-9428-efcb8a5a2471", "content-length": "691", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
179522025-11-12T20:54:05.696ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
179532025-11-12T20:54:05.696ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
179542025-11-12T20:54:05.696ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
179552025-11-12T20:54:05.698ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 420.647236ms iteration = 18 status = {}
179562025-11-12T20:54:05.704ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "625034f3-79ca-443d-a075-4adc71da38c4", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
179572025-11-12T20:54:05.704ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
179582025-11-12T20:54:05.704ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
179592025-11-12T20:54:05.711ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e5c4174-3a96-4240-8d3f-dea473cb83b3", "content-length": "682", "date": "Wed, 12 Nov 2025 20:54:05 GMT"} })
179602025-11-12T20:54:05.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179612025-11-12T20:54:05.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179622025-11-12T20:54:05.715ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179632025-11-12T20:54:05.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179642025-11-12T20:54:05.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179652025-11-12T20:54:05.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179662025-11-12T20:54:05.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
179672025-11-12T20:54:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
179682025-11-12T20:54:05.728ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 450.043147ms iteration = 18 status = {}
179692025-11-12T20:54:06.131ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
179702025-11-12T20:54:06.131ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
179712025-11-12T20:54:06.162ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
179722025-11-12T20:54:06.162ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:43199
179732025-11-12T20:54:06.162ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43199 req_id = 8305e1dc-aaee-4579-8e41-b4f5a0e2945e uri = /metrics/producers version = 1.0.0
179742025-11-12T20:54:06.162ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43199 req_id = 8305e1dc-aaee-4579-8e41-b4f5a0e2945e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
179752025-11-12T20:54:06.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43199 req_id = 8305e1dc-aaee-4579-8e41-b4f5a0e2945e resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
179762025-11-12T20:54:06.240ZINFOnexus: assigned collector to new producer collector_id = ecb0f932-e2e8-41d3-85a9-494e5cae5bce file = nexus/src/app/oximeter.rs:76 producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
179772025-11-12T20:54:06.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 77750 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:43199 req_id = 8305e1dc-aaee-4579-8e41-b4f5a0e2945e response_code = 201 uri = /metrics/producers
179782025-11-12T20:54:06.240ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "8305e1dc-aaee-4579-8e41-b4f5a0e2945e", "content-length": "41", "date": "Wed, 12 Nov 2025 20:54:06 GMT"} })
179792025-11-12T20:54:06.242ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
179802025-11-12T20:54:07.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 14327a49-dc13-4f1a-9119-11af8ea13ef7 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
179812025-11-12T20:54:10.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 104 reason = Timeout
179822025-11-12T20:54:10.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
179832025-11-12T20:54:10.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
179842025-11-12T20:54:10.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179852025-11-12T20:54:10.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
179862025-11-12T20:54:10.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
179872025-11-12T20:54:10.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
179882025-11-12T20:54:10.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
179892025-11-12T20:54:10.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.453133ms iteration = 104 status = {"config_updated":false}
179902025-11-12T20:54:15.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
179912025-11-12T20:54:15.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
179922025-11-12T20:54:15.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
179932025-11-12T20:54:15.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 105 reason = Timeout
179942025-11-12T20:54:15.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
179952025-11-12T20:54:15.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
179962025-11-12T20:54:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
179972025-11-12T20:54:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
179982025-11-12T20:54:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
179992025-11-12T20:54:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180002025-11-12T20:54:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
180012025-11-12T20:54:15.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
180022025-11-12T20:54:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
180032025-11-12T20:54:15.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
180042025-11-12T20:54:15.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.226777ms iteration = 105 status = {"config_updated":false}
180052025-11-12T20:54:15.313ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
180062025-11-12T20:54:15.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
180072025-11-12T20:54:15.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180082025-11-12T20:54:15.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
180092025-11-12T20:54:15.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180102025-11-12T20:54:15.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180112025-11-12T20:54:15.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180122025-11-12T20:54:15.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180132025-11-12T20:54:15.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180142025-11-12T20:54:15.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
180152025-11-12T20:54:15.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180162025-11-12T20:54:15.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180172025-11-12T20:54:15.346ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
180182025-11-12T20:54:15.346ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.800455ms iteration = 53 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
180192025-11-12T20:54:15.348ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.669613ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
180202025-11-12T20:54:17.915ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:34179
180212025-11-12T20:54:17.915ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 1d76c9b4-88da-4720-8322-9f5c18b2f4c1 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
180222025-11-12T20:54:17.915ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 1d76c9b4-88da-4720-8322-9f5c18b2f4c1 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
180232025-11-12T20:54:17.916ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 1d76c9b4-88da-4720-8322-9f5c18b2f4c1 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
180242025-11-12T20:54:17.918ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2865 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 1d76c9b4-88da-4720-8322-9f5c18b2f4c1 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
180252025-11-12T20:54:17.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = e9d77ae9-f1c4-458a-9415-5513fb14d770 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
180262025-11-12T20:54:17.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = e9d77ae9-f1c4-458a-9415-5513fb14d770 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
180272025-11-12T20:54:17.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = e9d77ae9-f1c4-458a-9415-5513fb14d770 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
180282025-11-12T20:54:17.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2569 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = e9d77ae9-f1c4-458a-9415-5513fb14d770 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
180292025-11-12T20:54:17.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 288 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 4f976545-bd50-4a5d-a64d-1bb2a03c2321 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
180302025-11-12T20:54:20.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 46 reason = Timeout
180312025-11-12T20:54:20.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
180322025-11-12T20:54:20.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
180332025-11-12T20:54:20.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 106 reason = Timeout
180342025-11-12T20:54:20.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
180352025-11-12T20:54:20.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
180362025-11-12T20:54:20.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180372025-11-12T20:54:20.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180382025-11-12T20:54:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
180392025-11-12T20:54:20.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
180402025-11-12T20:54:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
180412025-11-12T20:54:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
180422025-11-12T20:54:20.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
180432025-11-12T20:54:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
180442025-11-12T20:54:20.284ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e3133932-a041-4d2e-804a-0de86a112082 original_time_started = 2025-11-12 20:53:36.623549 UTC
180452025-11-12T20:54:20.284ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.188996ms iteration = 46 status = {"Loaded":{"collection_id":"e3133932-a041-4d2e-804a-0de86a112082","time_loaded":"2025-11-12T20:54:20.277185381Z","time_started":"2025-11-12T20:53:36.623549Z"}}
180462025-11-12T20:54:20.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 36 reason = Timeout
180472025-11-12T20:54:20.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
180482025-11-12T20:54:20.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
180492025-11-12T20:54:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.155995ms iteration = 106 status = {"config_updated":false}
180502025-11-12T20:54:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
180512025-11-12T20:54:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180522025-11-12T20:54:20.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
180532025-11-12T20:54:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
180542025-11-12T20:54:20.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
180552025-11-12T20:54:20.290ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 4.955648ms iteration = 36 status = "NoSitrep"
180562025-11-12T20:54:20.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 36 reason = Timeout
180572025-11-12T20:54:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
180582025-11-12T20:54:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
180592025-11-12T20:54:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
180602025-11-12T20:54:20.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
180612025-11-12T20:54:20.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180622025-11-12T20:54:20.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
180632025-11-12T20:54:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
180642025-11-12T20:54:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
180652025-11-12T20:54:20.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
180662025-11-12T20:54:20.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
180672025-11-12T20:54:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180682025-11-12T20:54:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
180692025-11-12T20:54:20.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
180702025-11-12T20:54:20.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.282146ms iteration = 36 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
180712025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
180722025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
180732025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180742025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
180752025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 107 reason = Timeout
180762025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
180772025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
180782025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180792025-11-12T20:54:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
180802025-11-12T20:54:25.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180812025-11-12T20:54:25.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180822025-11-12T20:54:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
180832025-11-12T20:54:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180842025-11-12T20:54:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180852025-11-12T20:54:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
180862025-11-12T20:54:25.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
180872025-11-12T20:54:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
180882025-11-12T20:54:25.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
180892025-11-12T20:54:25.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.529224ms iteration = 107 status = {"config_updated":false}
180902025-11-12T20:54:25.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180912025-11-12T20:54:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180922025-11-12T20:54:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180932025-11-12T20:54:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
180942025-11-12T20:54:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
180952025-11-12T20:54:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
180962025-11-12T20:54:25.310ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
180972025-11-12T20:54:25.310ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.45764ms iteration = 54 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
180982025-11-12T20:54:25.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
180992025-11-12T20:54:25.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
181002025-11-12T20:54:25.355ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.424164ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
181012025-11-12T20:54:27.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 324 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 5ba73e41-701f-4aef-ad98-fd07b43e1d5b response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
181022025-11-12T20:54:30.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 108 reason = Timeout
181032025-11-12T20:54:30.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
181042025-11-12T20:54:30.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
181052025-11-12T20:54:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181062025-11-12T20:54:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
181072025-11-12T20:54:30.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
181082025-11-12T20:54:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
181092025-11-12T20:54:30.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
181102025-11-12T20:54:30.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.767955ms iteration = 108 status = {"config_updated":false}
181112025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
181122025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
181132025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
181142025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
181152025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
181162025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
181172025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
181182025-11-12T20:54:35.275ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
181192025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
181202025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
181212025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
181222025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
181232025-11-12T20:54:35.275ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = ExternalNetworking step id = Ensure
181242025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
181252025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
181262025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
181272025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181282025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
181292025-11-12T20:54:35.275ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:44:35.275580686 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
181302025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
181312025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
181322025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
181332025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
181342025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
181352025-11-12T20:54:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
181362025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
181372025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
181382025-11-12T20:54:35.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1470, tv_nsec: 772472798 } })
181392025-11-12T20:54:35.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1470, tv_nsec: 772500818 } })
181402025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
181412025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
181422025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
181432025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181442025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
181452025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 47 reason = Timeout
181462025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
181472025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
181482025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.430697ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
181492025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
181502025-11-12T20:54:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.350605ms iteration = 10 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
181512025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
181522025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
181532025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
181542025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
181552025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
181562025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
181572025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
181582025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
181592025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
181602025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
181612025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
181622025-11-12T20:54:35.277ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
181632025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
181642025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
181652025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
181662025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
181672025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
181682025-11-12T20:54:35.278ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
181692025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
181702025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
181712025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
181722025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
181732025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
181742025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
181752025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
181762025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
181772025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
181782025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 23 reason = Timeout
181792025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
181802025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
181812025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
181822025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
181832025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
181842025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 109 reason = Timeout
181852025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
181862025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
181872025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
181882025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
181892025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
181902025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
181912025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
181922025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
181932025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
181942025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
181952025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
181962025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
181972025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
181982025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
181992025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
182002025-11-12T20:54:35.278ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
182012025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
182022025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
182032025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
182042025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
182052025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
182062025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
182072025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
182082025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
182092025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
182102025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
182112025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
182122025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
182132025-11-12T20:54:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
182142025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
182152025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
182162025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182172025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
182182025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
182192025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
182202025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
182212025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182222025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
182232025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
182242025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
182252025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
182262025-11-12T20:54:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1470, tv_nsec: 775129516 } })
182272025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
182282025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
182292025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
182302025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
182312025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
182322025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
182332025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
182342025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
182352025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
182362025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
182372025-11-12T20:54:35.279ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182382025-11-12T20:54:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1470, tv_nsec: 776257716 } })
182392025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
182402025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
182412025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
182422025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
182432025-11-12T20:54:35.280ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:44:35.275580686 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
182442025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.02871ms iteration = 10 status = {"expiration":"2025-11-12T20:44:35.275580686Z","pruned":[]}
182452025-11-12T20:54:35.280ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1470, tv_nsec: 776909102 } })
182462025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
182472025-11-12T20:54:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
182482025-11-12T20:54:35.281ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
182492025-11-12T20:54:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
182502025-11-12T20:54:35.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182512025-11-12T20:54:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182522025-11-12T20:54:35.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182532025-11-12T20:54:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182542025-11-12T20:54:35.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
182552025-11-12T20:54:35.284ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
182562025-11-12T20:54:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
182572025-11-12T20:54:35.285ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
182582025-11-12T20:54:35.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 37 reason = Timeout
182592025-11-12T20:54:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 6.546589ms iteration = 10 status = {"by_rx":{},"error":null}
182602025-11-12T20:54:35.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
182612025-11-12T20:54:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
182622025-11-12T20:54:35.285ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182632025-11-12T20:54:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.508422ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
182642025-11-12T20:54:35.287ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
182652025-11-12T20:54:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
182662025-11-12T20:54:35.287ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
182672025-11-12T20:54:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
182682025-11-12T20:54:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
182692025-11-12T20:54:35.287ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.537496ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
182702025-11-12T20:54:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
182712025-11-12T20:54:35.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
182722025-11-12T20:54:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
182732025-11-12T20:54:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
182742025-11-12T20:54:35.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
182752025-11-12T20:54:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
182762025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182772025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 10 reason = Timeout
182782025-11-12T20:54:35.291ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
182792025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
182802025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 37 reason = Timeout
182812025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
182822025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
182832025-11-12T20:54:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
182842025-11-12T20:54:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
182852025-11-12T20:54:35.294ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
182862025-11-12T20:54:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
182872025-11-12T20:54:35.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
182882025-11-12T20:54:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
182892025-11-12T20:54:35.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
182902025-11-12T20:54:35.301ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182912025-11-12T20:54:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
182922025-11-12T20:54:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
182932025-11-12T20:54:35.305ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182942025-11-12T20:54:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182952025-11-12T20:54:35.360ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182962025-11-12T20:54:35.363ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182972025-11-12T20:54:35.364ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
182982025-11-12T20:54:35.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
182992025-11-12T20:54:35.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
183002025-11-12T20:54:35.370ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183012025-11-12T20:54:35.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
183022025-11-12T20:54:35.373ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183032025-11-12T20:54:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
183042025-11-12T20:54:35.374ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
183052025-11-12T20:54:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
183062025-11-12T20:54:35.376ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183072025-11-12T20:54:35.376ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
183082025-11-12T20:54:35.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
183092025-11-12T20:54:35.377ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
183102025-11-12T20:54:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
183112025-11-12T20:54:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
183122025-11-12T20:54:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
183132025-11-12T20:54:35.380ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
183142025-11-12T20:54:35.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
183152025-11-12T20:54:35.381ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183162025-11-12T20:54:35.383ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183172025-11-12T20:54:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
183182025-11-12T20:54:35.385ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
183192025-11-12T20:54:35.385ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 107.512052ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
183202025-11-12T20:54:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
183212025-11-12T20:54:35.388ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183222025-11-12T20:54:35.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183232025-11-12T20:54:35.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183242025-11-12T20:54:35.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
183252025-11-12T20:54:35.388ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
183262025-11-12T20:54:35.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
183272025-11-12T20:54:35.390ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183282025-11-12T20:54:35.392ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183292025-11-12T20:54:35.392ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
183302025-11-12T20:54:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
183312025-11-12T20:54:35.392ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
183322025-11-12T20:54:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
183332025-11-12T20:54:35.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
183342025-11-12T20:54:35.393ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
183352025-11-12T20:54:35.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
183362025-11-12T20:54:35.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
183372025-11-12T20:54:35.394ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
183382025-11-12T20:54:35.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
183392025-11-12T20:54:35.395ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
183402025-11-12T20:54:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
183412025-11-12T20:54:35.395ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183422025-11-12T20:54:35.396ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
183432025-11-12T20:54:35.396ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
183442025-11-12T20:54:35.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
183452025-11-12T20:54:35.397ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
183462025-11-12T20:54:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
183472025-11-12T20:54:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
183482025-11-12T20:54:35.399ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
183492025-11-12T20:54:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
183502025-11-12T20:54:35.399ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e3133932-a041-4d2e-804a-0de86a112082 original_time_started = 2025-11-12 20:53:36.623549 UTC
183512025-11-12T20:54:35.399ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 123.021813ms iteration = 47 status = {"Loaded":{"collection_id":"e3133932-a041-4d2e-804a-0de86a112082","time_loaded":"2025-11-12T20:54:35.276782647Z","time_started":"2025-11-12T20:53:36.623549Z"}}
183522025-11-12T20:54:35.400ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183532025-11-12T20:54:35.400ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183542025-11-12T20:54:35.401ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 123.348562ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
183552025-11-12T20:54:35.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 123.700391ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
183562025-11-12T20:54:35.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183572025-11-12T20:54:35.403ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1470, tv_nsec: 899845904 } })
183582025-11-12T20:54:35.404ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
183592025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
183602025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
183612025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
183622025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
183632025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
183642025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
183652025-11-12T20:54:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183662025-11-12T20:54:35.406ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183672025-11-12T20:54:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
183682025-11-12T20:54:35.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
183692025-11-12T20:54:35.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
183702025-11-12T20:54:35.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183712025-11-12T20:54:35.410ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
183722025-11-12T20:54:35.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183732025-11-12T20:54:35.411ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183742025-11-12T20:54:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183752025-11-12T20:54:35.411ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
183762025-11-12T20:54:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 132.94721ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
183772025-11-12T20:54:35.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
183782025-11-12T20:54:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
183792025-11-12T20:54:35.413ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183802025-11-12T20:54:35.414ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 129.05621ms iteration = 37 status = "NoSitrep"
183812025-11-12T20:54:35.414ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183822025-11-12T20:54:35.416ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 137.795906ms iteration = 5 status = {"nsuccess":5}
183832025-11-12T20:54:35.416ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183842025-11-12T20:54:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
183852025-11-12T20:54:35.417ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
183862025-11-12T20:54:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
183872025-11-12T20:54:35.418ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
183882025-11-12T20:54:35.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183892025-11-12T20:54:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
183902025-11-12T20:54:35.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
183912025-11-12T20:54:35.419ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
183922025-11-12T20:54:35.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
183932025-11-12T20:54:35.421ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
183942025-11-12T20:54:35.421ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
183952025-11-12T20:54:35.421ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
183962025-11-12T20:54:35.422ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
183972025-11-12T20:54:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
183982025-11-12T20:54:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
183992025-11-12T20:54:35.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184002025-11-12T20:54:35.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184012025-11-12T20:54:35.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184022025-11-12T20:54:35.425ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
184032025-11-12T20:54:35.425ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
184042025-11-12T20:54:35.425ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 147.293042ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
184052025-11-12T20:54:35.427ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
184062025-11-12T20:54:35.429ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
184072025-11-12T20:54:35.429ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 151.06397ms iteration = 109 status = {"config_updated":false}
184082025-11-12T20:54:35.430ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
184092025-11-12T20:54:35.432ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
184102025-11-12T20:54:35.433ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
184112025-11-12T20:54:35.434ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
184122025-11-12T20:54:35.436ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
184132025-11-12T20:54:35.447ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
184142025-11-12T20:54:35.448ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
184152025-11-12T20:54:35.448ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 170.361739ms iteration = 55 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
184162025-11-12T20:54:35.449ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
184172025-11-12T20:54:35.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
184182025-11-12T20:54:35.451ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
184192025-11-12T20:54:35.456ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
184202025-11-12T20:54:35.457ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
184212025-11-12T20:54:35.457ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
184222025-11-12T20:54:35.457ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
184232025-11-12T20:54:35.457ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
184242025-11-12T20:54:35.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184252025-11-12T20:54:35.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
184262025-11-12T20:54:35.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184272025-11-12T20:54:35.459ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
184282025-11-12T20:54:35.460ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184292025-11-12T20:54:35.461ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
184302025-11-12T20:54:35.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184312025-11-12T20:54:35.462ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
184322025-11-12T20:54:35.466ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184332025-11-12T20:54:35.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184342025-11-12T20:54:35.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184352025-11-12T20:54:35.466ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
184362025-11-12T20:54:35.466ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184372025-11-12T20:54:35.466ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
184382025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 175.853101ms iteration = 37 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
184392025-11-12T20:54:35.467ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = SledList step id = Fetch
184402025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184412025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
184422025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
184432025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
184442025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
184452025-11-12T20:54:35.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184462025-11-12T20:54:35.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
184472025-11-12T20:54:35.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184482025-11-12T20:54:35.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184492025-11-12T20:54:35.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184502025-11-12T20:54:35.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
184512025-11-12T20:54:35.473ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
184522025-11-12T20:54:35.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
184532025-11-12T20:54:35.474ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184542025-11-12T20:54:35.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184552025-11-12T20:54:35.475ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 197.209303ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
184562025-11-12T20:54:35.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184572025-11-12T20:54:35.475ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
184582025-11-12T20:54:35.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184592025-11-12T20:54:35.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184602025-11-12T20:54:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
184612025-11-12T20:54:35.477ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
184622025-11-12T20:54:35.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184632025-11-12T20:54:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
184642025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
184652025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
184662025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
184672025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
184682025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
184692025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
184702025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
184712025-11-12T20:54:35.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184722025-11-12T20:54:35.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184732025-11-12T20:54:35.483ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184742025-11-12T20:54:35.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184752025-11-12T20:54:35.487ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184762025-11-12T20:54:35.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
184772025-11-12T20:54:35.487ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
184782025-11-12T20:54:35.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
184792025-11-12T20:54:35.488ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
184802025-11-12T20:54:35.489ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
184812025-11-12T20:54:35.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
184822025-11-12T20:54:35.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
184832025-11-12T20:54:35.489ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
184842025-11-12T20:54:35.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
184852025-11-12T20:54:35.490ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
184862025-11-12T20:54:35.490ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 211.887153ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
184872025-11-12T20:54:35.491ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 213.759882ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
184882025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
184892025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
184902025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
184912025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 217.528529ms iteration = 10 status = {"generation":1}
184922025-11-12T20:54:35.493ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
184932025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
184942025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
184952025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
184962025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
184972025-11-12T20:54:35.493ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
184982025-11-12T20:54:35.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
184992025-11-12T20:54:35.497ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 218.915615ms iteration = 19 status = 0
185002025-11-12T20:54:35.498ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 219.561572ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
185012025-11-12T20:54:35.498ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
185022025-11-12T20:54:35.498ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
185032025-11-12T20:54:35.498ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 219.792008ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
185042025-11-12T20:54:35.498ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
185052025-11-12T20:54:35.498ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
185062025-11-12T20:54:35.498ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0831864-f430-4247-ab6b-38ffe76b27ad", "content-length": "26", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
185072025-11-12T20:54:35.498ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
185082025-11-12T20:54:35.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
185092025-11-12T20:54:35.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
185102025-11-12T20:54:35.503ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
185112025-11-12T20:54:35.511ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
185122025-11-12T20:54:35.511ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 232.868406ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
185132025-11-12T20:54:35.519ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
185142025-11-12T20:54:35.521ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 246.206781ms iteration = 19 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
185152025-11-12T20:54:35.533ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
185162025-11-12T20:54:35.533ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
185172025-11-12T20:54:35.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
185182025-11-12T20:54:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
185192025-11-12T20:54:35.541ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185202025-11-12T20:54:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
185212025-11-12T20:54:35.548ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
185222025-11-12T20:54:35.548ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
185232025-11-12T20:54:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
185242025-11-12T20:54:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
185252025-11-12T20:54:35.556ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
185262025-11-12T20:54:35.556ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
185272025-11-12T20:54:35.556ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
185282025-11-12T20:54:35.556ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 280.695314ms iteration = 10 status = {"generation":2}
185292025-11-12T20:54:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185302025-11-12T20:54:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185312025-11-12T20:54:35.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185322025-11-12T20:54:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185332025-11-12T20:54:35.566ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
185342025-11-12T20:54:35.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185352025-11-12T20:54:35.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
185362025-11-12T20:54:35.569ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
185372025-11-12T20:54:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
185382025-11-12T20:54:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
185392025-11-12T20:54:35.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185402025-11-12T20:54:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185412025-11-12T20:54:35.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185422025-11-12T20:54:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185432025-11-12T20:54:35.579ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
185442025-11-12T20:54:35.579ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
185452025-11-12T20:54:35.579ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
185462025-11-12T20:54:35.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185472025-11-12T20:54:35.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
185482025-11-12T20:54:35.580ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
185492025-11-12T20:54:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
185502025-11-12T20:54:35.583ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185512025-11-12T20:54:35.585ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
185522025-11-12T20:54:35.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185532025-11-12T20:54:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185542025-11-12T20:54:35.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
185552025-11-12T20:54:35.587ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
185562025-11-12T20:54:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
185572025-11-12T20:54:35.588ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185582025-11-12T20:54:35.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185592025-11-12T20:54:35.592ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
185602025-11-12T20:54:35.592ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
185612025-11-12T20:54:35.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
185622025-11-12T20:54:35.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
185632025-11-12T20:54:35.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185642025-11-12T20:54:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185652025-11-12T20:54:35.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185662025-11-12T20:54:35.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185672025-11-12T20:54:35.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185682025-11-12T20:54:35.612ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
185692025-11-12T20:54:35.612ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "f20f068a-a6dc-4ab2-b994-85268e38301e", "content-length": "78", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
185702025-11-12T20:54:35.612ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
185712025-11-12T20:54:35.612ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
185722025-11-12T20:54:35.612ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
185732025-11-12T20:54:35.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
185742025-11-12T20:54:35.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
185752025-11-12T20:54:35.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
185762025-11-12T20:54:35.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185772025-11-12T20:54:35.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185782025-11-12T20:54:35.617ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
185792025-11-12T20:54:35.617ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
185802025-11-12T20:54:35.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
185812025-11-12T20:54:35.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185822025-11-12T20:54:35.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185832025-11-12T20:54:35.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
185842025-11-12T20:54:35.620ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
185852025-11-12T20:54:35.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
185862025-11-12T20:54:35.620ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
185872025-11-12T20:54:35.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
185882025-11-12T20:54:35.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
185892025-11-12T20:54:35.622ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
185902025-11-12T20:54:35.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185912025-11-12T20:54:35.624ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
185922025-11-12T20:54:35.624ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
185932025-11-12T20:54:35.625ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4279c998-69e0-4cef-bc57-8e459b9d4e01", "content-length": "26", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
185942025-11-12T20:54:35.625ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
185952025-11-12T20:54:35.626ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
185962025-11-12T20:54:35.626ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
185972025-11-12T20:54:35.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
185982025-11-12T20:54:35.627ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
185992025-11-12T20:54:35.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
186002025-11-12T20:54:35.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186012025-11-12T20:54:35.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186022025-11-12T20:54:35.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186032025-11-12T20:54:35.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186042025-11-12T20:54:35.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186052025-11-12T20:54:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
186062025-11-12T20:54:35.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186072025-11-12T20:54:35.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186082025-11-12T20:54:35.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 366.829873ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
186092025-11-12T20:54:35.646ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 368.230349ms iteration = 30 status = {"physical_disks_added":0}
186102025-11-12T20:54:35.649ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = DeployNexusRecords step id = Ensure
186112025-11-12T20:54:35.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
186122025-11-12T20:54:35.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
186132025-11-12T20:54:35.656ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
186142025-11-12T20:54:35.656ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
186152025-11-12T20:54:35.656ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 378.222608ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
186162025-11-12T20:54:35.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186172025-11-12T20:54:35.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186182025-11-12T20:54:35.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186192025-11-12T20:54:35.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186202025-11-12T20:54:35.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186212025-11-12T20:54:35.687ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
186222025-11-12T20:54:35.687ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
186232025-11-12T20:54:35.687ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "036b0392-3b21-417a-963d-024efaecce19", "content-length": "81", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
186242025-11-12T20:54:35.690ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
186252025-11-12T20:54:35.690ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
186262025-11-12T20:54:35.690ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e430799-2ce1-4eb8-930d-715b752058cb", "content-length": "81", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
186272025-11-12T20:54:35.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
186282025-11-12T20:54:35.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186292025-11-12T20:54:35.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186302025-11-12T20:54:35.693ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
186312025-11-12T20:54:35.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186322025-11-12T20:54:35.695ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
186332025-11-12T20:54:35.695ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
186342025-11-12T20:54:35.696ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "bee7ad2f-3a90-495b-a58c-84c671e9c9e6", "content-length": "81", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
186352025-11-12T20:54:35.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
186362025-11-12T20:54:35.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186372025-11-12T20:54:35.699ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 421.469687ms iteration = 19 status = {"disabled":false,"errors":[],"sps":[]}
186382025-11-12T20:54:35.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186392025-11-12T20:54:35.699ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
186402025-11-12T20:54:35.699ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 424.31018ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
186412025-11-12T20:54:35.700ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
186422025-11-12T20:54:35.700ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d315c657-4acd-46f8-b6f9-a95d0f7dbbc4", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
186432025-11-12T20:54:35.700ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
186442025-11-12T20:54:35.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186452025-11-12T20:54:35.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186462025-11-12T20:54:35.703ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f8a0bb80-0e01-457c-b418-890ca06cd983", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
186472025-11-12T20:54:35.703ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
186482025-11-12T20:54:35.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186492025-11-12T20:54:35.705ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
186502025-11-12T20:54:35.705ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 430.516161ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
186512025-11-12T20:54:35.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186522025-11-12T20:54:35.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186532025-11-12T20:54:35.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
186542025-11-12T20:54:35.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186552025-11-12T20:54:35.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186562025-11-12T20:54:35.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186572025-11-12T20:54:35.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186582025-11-12T20:54:35.723ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
186592025-11-12T20:54:35.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
186602025-11-12T20:54:35.723ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
186612025-11-12T20:54:35.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186622025-11-12T20:54:35.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
186632025-11-12T20:54:35.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186642025-11-12T20:54:35.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
186652025-11-12T20:54:35.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
186662025-11-12T20:54:35.731ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 455.650362ms iteration = 19 status = {}
186672025-11-12T20:54:35.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186682025-11-12T20:54:35.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186692025-11-12T20:54:35.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186702025-11-12T20:54:35.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186712025-11-12T20:54:35.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
186722025-11-12T20:54:35.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186732025-11-12T20:54:35.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186742025-11-12T20:54:35.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
186752025-11-12T20:54:35.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186762025-11-12T20:54:35.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186772025-11-12T20:54:35.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186782025-11-12T20:54:35.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186792025-11-12T20:54:35.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186802025-11-12T20:54:35.749ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
186812025-11-12T20:54:35.749ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
186822025-11-12T20:54:35.750ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5d75a08-9909-4c6d-93b6-0b99e5894fb0", "content-length": "372", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
186832025-11-12T20:54:35.750ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
186842025-11-12T20:54:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
186852025-11-12T20:54:35.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
186862025-11-12T20:54:35.756ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
186872025-11-12T20:54:35.756ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 478.499233ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
186882025-11-12T20:54:35.760ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
186892025-11-12T20:54:35.760ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "4e1caa95-96c9-45f2-bf56-de6e54a63bf5", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
186902025-11-12T20:54:35.760ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
186912025-11-12T20:54:35.763ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
186922025-11-12T20:54:35.763ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 472.179879ms iteration = 10 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
186932025-11-12T20:54:35.770ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
186942025-11-12T20:54:35.770ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
186952025-11-12T20:54:35.771ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "80e5bb5e-7b4f-48e6-9c92-39300b62fd21", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
186962025-11-12T20:54:35.771ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
186972025-11-12T20:54:35.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
186982025-11-12T20:54:35.774ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
186992025-11-12T20:54:35.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
187002025-11-12T20:54:35.776ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187012025-11-12T20:54:35.776ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187022025-11-12T20:54:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187032025-11-12T20:54:35.779ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
187042025-11-12T20:54:35.779ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
187052025-11-12T20:54:35.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
187062025-11-12T20:54:35.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187072025-11-12T20:54:35.780ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187082025-11-12T20:54:35.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187092025-11-12T20:54:35.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
187102025-11-12T20:54:35.783ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
187112025-11-12T20:54:35.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
187122025-11-12T20:54:35.784ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
187132025-11-12T20:54:35.784ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
187142025-11-12T20:54:35.785ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "674b5f65-9ff1-4f04-9e1b-2d0d9fda55a3", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
187152025-11-12T20:54:35.786ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 282961989 } })
187162025-11-12T20:54:35.787ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = SledAgent step id = Ensure
187172025-11-12T20:54:35.787ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 509.557367ms iteration = 28 status = {}
187182025-11-12T20:54:35.787ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Dependency
187192025-11-12T20:54:35.787ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
187202025-11-12T20:54:35.787ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187212025-11-12T20:54:35.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187222025-11-12T20:54:35.789ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187232025-11-12T20:54:35.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
187242025-11-12T20:54:35.793ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187252025-11-12T20:54:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187262025-11-12T20:54:35.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187272025-11-12T20:54:35.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
187282025-11-12T20:54:35.800ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187292025-11-12T20:54:35.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187302025-11-12T20:54:35.813ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
187312025-11-12T20:54:35.813ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a3cfc640-5075-4032-8033-9092806e652d", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187322025-11-12T20:54:35.813ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187332025-11-12T20:54:35.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187342025-11-12T20:54:35.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187352025-11-12T20:54:35.816ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1cd5eab2-bbce-4ae1-b77c-2b0dd9664903", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187362025-11-12T20:54:35.816ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187372025-11-12T20:54:35.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187382025-11-12T20:54:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187392025-11-12T20:54:35.818ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "307c739d-ab97-446b-bd6e-3fecb1f9bd8f", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187402025-11-12T20:54:35.818ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187412025-11-12T20:54:35.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187422025-11-12T20:54:35.820ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
187432025-11-12T20:54:35.820ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 545.154698ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
187442025-11-12T20:54:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
187452025-11-12T20:54:35.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187462025-11-12T20:54:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187472025-11-12T20:54:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187482025-11-12T20:54:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187492025-11-12T20:54:35.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187502025-11-12T20:54:35.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
187512025-11-12T20:54:35.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187522025-11-12T20:54:35.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187532025-11-12T20:54:35.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187542025-11-12T20:54:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187552025-11-12T20:54:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187562025-11-12T20:54:35.837ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
187572025-11-12T20:54:35.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187582025-11-12T20:54:35.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187592025-11-12T20:54:35.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187602025-11-12T20:54:35.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187612025-11-12T20:54:35.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187622025-11-12T20:54:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
187632025-11-12T20:54:35.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187642025-11-12T20:54:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187652025-11-12T20:54:35.848ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
187662025-11-12T20:54:35.848ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
187672025-11-12T20:54:35.849ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = e9c3eeba-31d4-429d-b512-a10d6773d588 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
187682025-11-12T20:54:35.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187692025-11-12T20:54:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187702025-11-12T20:54:35.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187712025-11-12T20:54:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
187722025-11-12T20:54:35.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187732025-11-12T20:54:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187742025-11-12T20:54:35.858ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
187752025-11-12T20:54:35.858ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
187762025-11-12T20:54:35.858ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 580.579633ms iteration = 23 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
187772025-11-12T20:54:35.890ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
187782025-11-12T20:54:35.893ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "ec24e0fa-fcd2-460e-a14b-40b8d8344b50", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
187792025-11-12T20:54:35.895ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = FirewallRules step id = Ensure
187802025-11-12T20:54:35.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
187812025-11-12T20:54:35.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
187822025-11-12T20:54:35.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
187832025-11-12T20:54:35.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
187842025-11-12T20:54:35.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
187852025-11-12T20:54:35.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
187862025-11-12T20:54:35.901ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
187872025-11-12T20:54:35.901ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
187882025-11-12T20:54:35.902ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "653b00d4-ce93-4368-a34e-c7c54507ef5a", "content-length": "372", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
187892025-11-12T20:54:35.902ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187902025-11-12T20:54:35.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187912025-11-12T20:54:35.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
187922025-11-12T20:54:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
187932025-11-12T20:54:35.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
187942025-11-12T20:54:35.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
187952025-11-12T20:54:35.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
187962025-11-12T20:54:35.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
187972025-11-12T20:54:35.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187982025-11-12T20:54:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
187992025-11-12T20:54:35.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188002025-11-12T20:54:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188012025-11-12T20:54:35.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188022025-11-12T20:54:35.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188032025-11-12T20:54:35.919ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
188042025-11-12T20:54:35.919ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
188052025-11-12T20:54:35.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188062025-11-12T20:54:35.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188072025-11-12T20:54:35.921ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b8bdfbf5-4dac-444d-b2ed-fd4b2d32968b", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
188082025-11-12T20:54:35.921ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
188092025-11-12T20:54:35.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188102025-11-12T20:54:35.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
188112025-11-12T20:54:35.924ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
188122025-11-12T20:54:35.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
188132025-11-12T20:54:35.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
188142025-11-12T20:54:35.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188152025-11-12T20:54:35.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188162025-11-12T20:54:35.927ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
188172025-11-12T20:54:35.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
188182025-11-12T20:54:35.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188192025-11-12T20:54:35.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188202025-11-12T20:54:35.931ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
188212025-11-12T20:54:35.931ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
188222025-11-12T20:54:35.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188232025-11-12T20:54:35.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188242025-11-12T20:54:35.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188252025-11-12T20:54:35.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188262025-11-12T20:54:35.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188272025-11-12T20:54:35.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188282025-11-12T20:54:35.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188292025-11-12T20:54:35.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188302025-11-12T20:54:35.939ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9e05c935-a2db-43c8-a3fe-b8d2b6b5eb5d", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
188312025-11-12T20:54:35.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188322025-11-12T20:54:35.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188332025-11-12T20:54:35.941ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 153.502342ms iteration = 29 status = {}
188342025-11-12T20:54:35.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188352025-11-12T20:54:35.945ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
188362025-11-12T20:54:35.946ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b6ba4fe-d65f-4517-8447-49d2b316b38e", "content-length": "691", "date": "Wed, 12 Nov 2025 20:54:35 GMT"} })
188372025-11-12T20:54:35.946ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
188382025-11-12T20:54:35.946ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
188392025-11-12T20:54:35.946ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
188402025-11-12T20:54:35.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
188412025-11-12T20:54:35.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188422025-11-12T20:54:35.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188432025-11-12T20:54:35.947ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 670.0499ms iteration = 19 status = {}
188442025-11-12T20:54:35.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188452025-11-12T20:54:35.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188462025-11-12T20:54:35.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188472025-11-12T20:54:35.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188482025-11-12T20:54:35.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188492025-11-12T20:54:35.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188502025-11-12T20:54:35.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188512025-11-12T20:54:35.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188522025-11-12T20:54:35.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188532025-11-12T20:54:35.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188542025-11-12T20:54:35.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
188552025-11-12T20:54:35.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
188562025-11-12T20:54:35.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188572025-11-12T20:54:35.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188582025-11-12T20:54:35.960ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
188592025-11-12T20:54:35.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
188602025-11-12T20:54:35.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188612025-11-12T20:54:35.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
188622025-11-12T20:54:35.965ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
188632025-11-12T20:54:35.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188642025-11-12T20:54:35.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188652025-11-12T20:54:35.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188662025-11-12T20:54:35.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188672025-11-12T20:54:35.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188682025-11-12T20:54:35.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188692025-11-12T20:54:35.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188702025-11-12T20:54:35.975ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
188712025-11-12T20:54:35.975ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
188722025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
188732025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
188742025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
188752025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
188762025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
188772025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
188782025-11-12T20:54:35.976ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
188792025-11-12T20:54:35.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188802025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
188812025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
188822025-11-12T20:54:35.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188832025-11-12T20:54:35.976ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
188842025-11-12T20:54:35.976ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
188852025-11-12T20:54:35.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
188862025-11-12T20:54:35.980ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
188872025-11-12T20:54:35.980ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
188882025-11-12T20:54:35.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188892025-11-12T20:54:35.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188902025-11-12T20:54:35.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188912025-11-12T20:54:35.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188922025-11-12T20:54:35.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188932025-11-12T20:54:35.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188942025-11-12T20:54:35.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188952025-11-12T20:54:35.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188962025-11-12T20:54:35.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188972025-11-12T20:54:35.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
188982025-11-12T20:54:35.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
188992025-11-12T20:54:35.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189002025-11-12T20:54:35.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189012025-11-12T20:54:35.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189022025-11-12T20:54:35.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189032025-11-12T20:54:36.003ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
189042025-11-12T20:54:36.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189052025-11-12T20:54:36.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189062025-11-12T20:54:36.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189072025-11-12T20:54:36.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189082025-11-12T20:54:36.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189092025-11-12T20:54:36.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189102025-11-12T20:54:36.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189112025-11-12T20:54:36.017ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
189122025-11-12T20:54:36.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189132025-11-12T20:54:36.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189142025-11-12T20:54:36.026ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
189152025-11-12T20:54:36.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189162025-11-12T20:54:36.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189172025-11-12T20:54:36.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189182025-11-12T20:54:36.031ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
189192025-11-12T20:54:36.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189202025-11-12T20:54:36.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189212025-11-12T20:54:36.033ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
189222025-11-12T20:54:36.033ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
189232025-11-12T20:54:36.033ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
189242025-11-12T20:54:36.106ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
189252025-11-12T20:54:36.106ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
189262025-11-12T20:54:36.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189272025-11-12T20:54:36.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189282025-11-12T20:54:36.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189292025-11-12T20:54:36.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189302025-11-12T20:54:36.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189312025-11-12T20:54:36.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
189322025-11-12T20:54:36.145ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
189332025-11-12T20:54:36.149ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "a53d7e70-6ad8-498b-aa11-22f6b9825feb", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
189342025-11-12T20:54:36.151ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
189352025-11-12T20:54:36.151ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = Dns step id = Ensure
189362025-11-12T20:54:36.151ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
189372025-11-12T20:54:36.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189382025-11-12T20:54:36.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189392025-11-12T20:54:36.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189402025-11-12T20:54:36.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189412025-11-12T20:54:36.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189422025-11-12T20:54:36.155ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "98202fdb-2162-4e8e-9e07-b33b23c102b4", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
189432025-11-12T20:54:36.155ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
189442025-11-12T20:54:36.155ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
189452025-11-12T20:54:36.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
189462025-11-12T20:54:36.161ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c7e4dec-afc0-4f51-afdb-f47264c3c33b", "content-length": "682", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
189472025-11-12T20:54:36.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
189482025-11-12T20:54:36.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
189492025-11-12T20:54:36.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
189502025-11-12T20:54:36.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
189512025-11-12T20:54:36.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
189522025-11-12T20:54:36.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
189532025-11-12T20:54:36.170ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
189542025-11-12T20:54:36.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
189552025-11-12T20:54:36.178ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 900.697758ms iteration = 19 status = {}
189562025-11-12T20:54:36.194ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
189572025-11-12T20:54:36.194ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
189582025-11-12T20:54:36.194ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
189592025-11-12T20:54:36.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189602025-11-12T20:54:36.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189612025-11-12T20:54:36.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189622025-11-12T20:54:36.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
189632025-11-12T20:54:36.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189642025-11-12T20:54:36.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189652025-11-12T20:54:36.201ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
189662025-11-12T20:54:36.201ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
189672025-11-12T20:54:36.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189682025-11-12T20:54:36.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189692025-11-12T20:54:36.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189702025-11-12T20:54:36.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
189712025-11-12T20:54:36.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189722025-11-12T20:54:36.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189732025-11-12T20:54:36.210ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
189742025-11-12T20:54:36.210ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
189752025-11-12T20:54:36.210ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
189762025-11-12T20:54:36.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189772025-11-12T20:54:36.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189782025-11-12T20:54:36.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189792025-11-12T20:54:36.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189802025-11-12T20:54:36.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189812025-11-12T20:54:36.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
189822025-11-12T20:54:36.218ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
189832025-11-12T20:54:36.218ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
189842025-11-12T20:54:36.218ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
189852025-11-12T20:54:36.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189862025-11-12T20:54:36.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189872025-11-12T20:54:36.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189882025-11-12T20:54:36.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
189892025-11-12T20:54:36.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189902025-11-12T20:54:36.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189912025-11-12T20:54:36.225ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
189922025-11-12T20:54:36.225ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
189932025-11-12T20:54:36.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189942025-11-12T20:54:36.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189952025-11-12T20:54:36.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189962025-11-12T20:54:36.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
189972025-11-12T20:54:36.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
189982025-11-12T20:54:36.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
189992025-11-12T20:54:36.232ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
190002025-11-12T20:54:36.232ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
190012025-11-12T20:54:36.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190022025-11-12T20:54:36.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190032025-11-12T20:54:36.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190042025-11-12T20:54:36.238ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
190052025-11-12T20:54:36.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190062025-11-12T20:54:36.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190072025-11-12T20:54:36.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190082025-11-12T20:54:36.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190092025-11-12T20:54:36.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190102025-11-12T20:54:36.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190112025-11-12T20:54:36.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190122025-11-12T20:54:36.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
190132025-11-12T20:54:36.247ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
190142025-11-12T20:54:36.247ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
190152025-11-12T20:54:36.247ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = OmicronZones step id = Cleanup
190162025-11-12T20:54:36.247ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = OmicronZones step id = Cleanup
190172025-11-12T20:54:36.247ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = PhysicalDisks step id = Cleanup
190182025-11-12T20:54:36.247ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = Clickhouse step id = Ensure
190192025-11-12T20:54:36.247ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = Clickhouse step id = Ensure
190202025-11-12T20:54:36.283ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
190212025-11-12T20:54:36.287ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "a6fb9b23-1bc7-4062-8dfd-9dc2640c3673", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190222025-11-12T20:54:36.289ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = SupportBundles step id = Cleanup
190232025-11-12T20:54:36.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190242025-11-12T20:54:36.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190252025-11-12T20:54:36.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190262025-11-12T20:54:36.292ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
190272025-11-12T20:54:36.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190282025-11-12T20:54:36.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190292025-11-12T20:54:36.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190302025-11-12T20:54:36.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190312025-11-12T20:54:36.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190322025-11-12T20:54:36.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
190332025-11-12T20:54:36.346ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = OmicronZones step id = Cleanup
190342025-11-12T20:54:36.346ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
190352025-11-12T20:54:36.346ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
190362025-11-12T20:54:36.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190372025-11-12T20:54:36.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190382025-11-12T20:54:36.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190392025-11-12T20:54:36.349ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
190402025-11-12T20:54:36.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190412025-11-12T20:54:36.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190422025-11-12T20:54:36.392ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
190432025-11-12T20:54:36.392ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
190442025-11-12T20:54:36.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = Cockroach step id = Ensure
190452025-11-12T20:54:36.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
190462025-11-12T20:54:36.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
190472025-11-12T20:54:36.411ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
190482025-11-12T20:54:36.411ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1cbcc1eb-5f2f-451f-9803-734ed6a5cac8 step component = MgsUpdates step id = Ensure
190492025-11-12T20:54:36.411ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Dependency
190502025-11-12T20:54:36.411ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
190512025-11-12T20:54:36.411ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Dependency
190522025-11-12T20:54:36.411ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
190532025-11-12T20:54:36.411ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.136338206s iteration = 11 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3830,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":191739192,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":191739132,"secs":0}},"event_index":1,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":191756592,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":181878257,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":181878206,"secs":0}},"event_index":2,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":373655060,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":137970400,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":137970360,"secs":0}},"event_index":3,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":511643060,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":108218630,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":108218580,"secs":0}},"event_index":4,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":619876021,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":256414586,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":256414546,"secs":0}},"event_index":5,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":876304887,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":96013125,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":96013095,"secs":0}},"event_index":6,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":972331572,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4280,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4240,"secs":0}},"event_index":7,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":972350222,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2000,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":1971,"secs":0}},"event_index":8,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":972354933,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3231,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3191,"secs":0}},"event_index":9,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":972360153,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1720,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1680,"secs":0}},"event_index":10,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":972363933,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41186086,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":41186026,"secs":0}},"event_index":11,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":13555609,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":56920243,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":56920183,"secs":0}},"event_index":12,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":70490002,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":46251907,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":46251856,"secs":0}},"event_index":13,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":116761350,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19173696,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19173646,"secs":0}},"event_index":14,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":135943876,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5170,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5120,"secs":0}},"event_index":15,"execution_id":"36f6600e-1eef-44cd-9f0a-6ddcd60fd424","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":135958126,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
190542025-11-12T20:54:36.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
190552025-11-12T20:54:36.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
190562025-11-12T20:54:36.413ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190572025-11-12T20:54:36.413ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190582025-11-12T20:54:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
190592025-11-12T20:54:36.416ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
190602025-11-12T20:54:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
190612025-11-12T20:54:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
190622025-11-12T20:54:36.416ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
190632025-11-12T20:54:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
190642025-11-12T20:54:36.459ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
190652025-11-12T20:54:36.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
190662025-11-12T20:54:36.461ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190672025-11-12T20:54:36.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
190682025-11-12T20:54:36.463ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
190692025-11-12T20:54:36.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
190702025-11-12T20:54:36.466ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
190712025-11-12T20:54:36.466ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 54.603703ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
190722025-11-12T20:54:36.481ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(f1d519c6-906b-463e-a78e-1e59b98d7eac, 0), (49918508-77d9-410a-ad8e-292b92856858, 0), (b503600e-eb36-40f3-9950-d4a90e646304, 0), (7f6facc3-32f0-422f-8146-b2467fc0279d, 0)] collection_id = f1d519c6-906b-463e-a78e-1e59b98d7eac
190732025-11-12T20:54:36.481ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
190742025-11-12T20:54:36.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
190752025-11-12T20:54:36.690ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = f1d519c6-906b-463e-a78e-1e59b98d7eac file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
190762025-11-12T20:54:36.690ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
190772025-11-12T20:54:36.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
190782025-11-12T20:54:36.876ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(b503600e-eb36-40f3-9950-d4a90e646304, 0), (7f6facc3-32f0-422f-8146-b2467fc0279d, 0), (e3133932-a041-4d2e-804a-0de86a112082, 0)]
190792025-11-12T20:54:36.876ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 373069979 } })
190802025-11-12T20:54:36.920ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 417086838 } })
190812025-11-12T20:54:36.921ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
190822025-11-12T20:54:36.921ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
190832025-11-12T20:54:36.921ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
190842025-11-12T20:54:36.922ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c733423-c148-4c6d-b091-381008751ec8", "content-length": "476", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190852025-11-12T20:54:36.922ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
190862025-11-12T20:54:36.922ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1031cb81-81a0-4952-8ccf-bbe86b371071", "content-length": "734", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190872025-11-12T20:54:36.922ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
190882025-11-12T20:54:36.923ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7d5220f-b64f-46c8-bdb4-96f4b78fcd31", "content-length": "111", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190892025-11-12T20:54:36.923ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
190902025-11-12T20:54:36.923ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd953726-ff21-4427-a6b0-d9149f99c5d3", "content-length": "111", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190912025-11-12T20:54:36.923ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
190922025-11-12T20:54:36.924ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6d0f6cb-c2a8-43a4-a590-89e98e2a02e8", "content-length": "163", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190932025-11-12T20:54:36.924ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
190942025-11-12T20:54:36.924ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3b01eeb-2500-437a-b563-a5d3b6ff6aec", "content-length": "163", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190952025-11-12T20:54:36.924ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
190962025-11-12T20:54:36.925ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e35441b-2cac-4fd1-b4db-0678d1c545f0", "content-length": "166", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190972025-11-12T20:54:36.925ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
190982025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70035220-b636-48c6-8b79-984659bde2ed", "content-length": "166", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
190992025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
191002025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e80f32c-962f-4325-99e5-8b64a6cde58e", "content-length": "702", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191012025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
191022025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1783da54-561c-4002-9677-c43eee3da647", "content-length": "727", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191032025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
191042025-11-12T20:54:36.926ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c82a8a31-0801-460c-a795-fa06294bb30b", "content-length": "729", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191052025-11-12T20:54:36.927ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
191062025-11-12T20:54:36.927ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b77d0f7b-49d3-4dbe-9c56-b53923e50c27", "content-length": "728", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191072025-11-12T20:54:36.927ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
191082025-11-12T20:54:36.927ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1abb1534-7b42-436e-a5f5-810e9ace6555", "content-length": "722", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191092025-11-12T20:54:36.927ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
191102025-11-12T20:54:36.927ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4504aa8-a384-4c88-adbb-3d3db9d2c330", "content-length": "10", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191112025-11-12T20:54:36.928ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
191122025-11-12T20:54:36.928ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd70f2e4-5b43-4ce9-a3d5-522cdb1440e5", "content-length": "144", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191132025-11-12T20:54:36.928ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
191142025-11-12T20:54:36.928ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb8f61f2-fc89-428e-884e-49d13873fa93", "content-length": "147", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191152025-11-12T20:54:36.928ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
191162025-11-12T20:54:36.929ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e81100d-be9c-4191-812a-58c3ae8c0aea", "content-length": "109", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191172025-11-12T20:54:36.929ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
191182025-11-12T20:54:36.929ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bf886bb-6426-47c8-8bd3-09bfee4d47bf", "content-length": "109", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191192025-11-12T20:54:36.929ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
191202025-11-12T20:54:36.930ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c1e8dfd-fa5d-4677-8ddc-eca2c4d8d28f", "content-length": "163", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191212025-11-12T20:54:36.930ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
191222025-11-12T20:54:36.930ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc76a707-2f3a-450d-a1a2-6ebaa1956618", "content-length": "163", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191232025-11-12T20:54:36.930ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
191242025-11-12T20:54:36.931ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "65b8c6f5-862b-4a28-91f3-500130eb4afb", "content-length": "166", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191252025-11-12T20:54:36.931ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
191262025-11-12T20:54:36.931ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0b0442e-8780-40a8-89c4-0c063b3a6f3c", "content-length": "166", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191272025-11-12T20:54:36.931ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
191282025-11-12T20:54:36.932ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19aaab7b-8289-4aa6-904f-5b5f53faf787", "content-length": "702", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191292025-11-12T20:54:36.932ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
191302025-11-12T20:54:36.932ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c2c2f0d-b153-4c04-8181-2c759bd1db83", "content-length": "727", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191312025-11-12T20:54:36.932ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
191322025-11-12T20:54:36.932ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec34f2b1-3132-4802-b51c-2fa93996873a", "content-length": "729", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191332025-11-12T20:54:36.932ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
191342025-11-12T20:54:36.933ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9917c448-1744-4e58-935c-4401204b4e71", "content-length": "728", "date": "Wed, 12 Nov 2025 20:54:36 GMT"} })
191352025-11-12T20:54:36.933ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
191362025-11-12T20:54:36.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
191372025-11-12T20:54:36.935ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191382025-11-12T20:54:36.939ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
191392025-11-12T20:54:36.939ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
191402025-11-12T20:54:36.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
191412025-11-12T20:54:36.941ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191422025-11-12T20:54:36.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
191432025-11-12T20:54:36.946ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
191442025-11-12T20:54:36.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
191452025-11-12T20:54:37.100ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
191462025-11-12T20:54:37.100ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
191472025-11-12T20:54:37.157ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "56bca09c-c6a5-46f0-9723-586bd26d9446", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191482025-11-12T20:54:37.160ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
191492025-11-12T20:54:37.160ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
191502025-11-12T20:54:37.160ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
191512025-11-12T20:54:37.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
191522025-11-12T20:54:37.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
191532025-11-12T20:54:37.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
191542025-11-12T20:54:37.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
191552025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
191562025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "130469e3-111d-482a-9441-9914ea29ddb9", "content-length": "64", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191572025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
191582025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6a07f6b-e035-4b90-ae86-6fde0ab98a7c", "content-length": "64", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191592025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11fbc53d-4fff-464a-b1c8-96eb4f29fd04", "content-length": "64", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191602025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "21569ada-218a-4729-b191-e8273e41a5cb", "content-length": "64", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191612025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
191622025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a210a76b-35a1-40df-ad1c-ea2dd86dc2f3", "content-length": "64", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191632025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
191642025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
191652025-11-12T20:54:37.161ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
191662025-11-12T20:54:37.204ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "55cdbf7f-c767-4199-a2a1-69301f0b6286", "content-length": "401626", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191672025-11-12T20:54:37.205ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "52e94116-6e1f-40b6-ba82-d1e1ec72e3ff", "content-length": "408719", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191682025-11-12T20:54:37.211ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "94408462-a996-41b2-a5c2-de8d8a1dba85", "content-length": "434383", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191692025-11-12T20:54:37.211ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "215fff19-2340-4196-a316-48c08a26ed81", "content-length": "404402", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191702025-11-12T20:54:37.216ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "48ddae3c-e403-43d0-84ca-4be2985dc466", "content-length": "405402", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191712025-11-12T20:54:37.219ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
191722025-11-12T20:54:37.219ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
191732025-11-12T20:54:37.219ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
191742025-11-12T20:54:37.219ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
191752025-11-12T20:54:37.219ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
191762025-11-12T20:54:37.251ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
191772025-11-12T20:54:37.251ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
191782025-11-12T20:54:37.254ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "207feddb-c071-4cda-a9de-f02f698800ad", "content-length": "124", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191792025-11-12T20:54:37.256ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
191802025-11-12T20:54:37.353ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
191812025-11-12T20:54:37.353ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
191822025-11-12T20:54:37.354ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "432ef2e5-cbf9-4f53-965d-1cd3934fe58f", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191832025-11-12T20:54:37.354ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
191842025-11-12T20:54:37.355ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
191852025-11-12T20:54:37.355ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
191862025-11-12T20:54:37.356ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "293b2f85-e6d8-47f0-9a57-c5cf766a2b6e", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191872025-11-12T20:54:37.356ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
191882025-11-12T20:54:37.358ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
191892025-11-12T20:54:37.359ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
191902025-11-12T20:54:37.359ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a4951e2-2f78-430b-8648-347f6d919e90", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:54:37 GMT"} })
191912025-11-12T20:54:37.359ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
191922025-11-12T20:54:37.362ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
191932025-11-12T20:54:37.362ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
191942025-11-12T20:54:37.364ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
191952025-11-12T20:54:37.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
191962025-11-12T20:54:37.366ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191972025-11-12T20:54:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
191982025-11-12T20:54:37.369ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
191992025-11-12T20:54:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
192002025-11-12T20:54:37.573ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 278c2b12-b876-45e5-9268-e4580c375342 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
192012025-11-12T20:54:37.573ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 278c2b12-b876-45e5-9268-e4580c375342 time_started = 2025-11-12 20:54:36.921675 UTC
192022025-11-12T20:54:37.573ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 48 reason = Dependency
192032025-11-12T20:54:37.573ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
192042025-11-12T20:54:37.573ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.162064032s iteration = 11 status = {"collection_id":"278c2b12-b876-45e5-9268-e4580c375342","time_done":"2025-11-12 20:54:37.362393 UTC","time_started":"2025-11-12 20:54:36.921675 UTC"}
192052025-11-12T20:54:37.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
192062025-11-12T20:54:37.575ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192072025-11-12T20:54:37.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
192082025-11-12T20:54:37.579ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
192092025-11-12T20:54:37.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
192102025-11-12T20:54:37.581ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
192112025-11-12T20:54:37.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
192122025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 24 reason = Dependency
192132025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 14 reason = Dependency
192142025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 50.139988ms iteration = 48 status = {"Loaded":{"collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","time_loaded":"2025-11-12T20:54:37.573567756Z","time_started":"2025-11-12T20:54:36.970966Z"}}
192152025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Dependency
192162025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192172025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192182025-11-12T20:54:37.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192192025-11-12T20:54:37.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192202025-11-12T20:54:37.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192212025-11-12T20:54:37.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192222025-11-12T20:54:37.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192232025-11-12T20:54:37.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192242025-11-12T20:54:37.625ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192252025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
192262025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192272025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
192282025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192292025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192302025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192312025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
192322025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192332025-11-12T20:54:37.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192342025-11-12T20:54:37.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192352025-11-12T20:54:37.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192362025-11-12T20:54:37.631ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192372025-11-12T20:54:37.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192382025-11-12T20:54:37.633ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192392025-11-12T20:54:37.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
192402025-11-12T20:54:37.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192412025-11-12T20:54:37.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
192422025-11-12T20:54:37.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192432025-11-12T20:54:37.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192442025-11-12T20:54:37.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192452025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192462025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192472025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
192482025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192492025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192502025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192512025-11-12T20:54:37.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192522025-11-12T20:54:37.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192532025-11-12T20:54:37.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192542025-11-12T20:54:37.641ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
192552025-11-12T20:54:37.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192562025-11-12T20:54:37.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192572025-11-12T20:54:37.641ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
192582025-11-12T20:54:37.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192592025-11-12T20:54:37.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192602025-11-12T20:54:37.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192612025-11-12T20:54:37.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192622025-11-12T20:54:37.646ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
192632025-11-12T20:54:37.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192642025-11-12T20:54:37.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192652025-11-12T20:54:37.647ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
192662025-11-12T20:54:37.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192672025-11-12T20:54:37.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192682025-11-12T20:54:37.653ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
192692025-11-12T20:54:37.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192702025-11-12T20:54:37.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192712025-11-12T20:54:37.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192722025-11-12T20:54:37.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
192732025-11-12T20:54:37.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192742025-11-12T20:54:37.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192752025-11-12T20:54:37.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192762025-11-12T20:54:37.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
192772025-11-12T20:54:37.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
192782025-11-12T20:54:37.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
192792025-11-12T20:54:37.664ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
192802025-11-12T20:54:37.664ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 41.006041ms iteration = 14 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
192812025-11-12T20:54:37.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192822025-11-12T20:54:37.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192832025-11-12T20:54:37.689ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 65.373132ms iteration = 31 status = {"physical_disks_added":0}
192842025-11-12T20:54:37.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192852025-11-12T20:54:37.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192862025-11-12T20:54:37.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192872025-11-12T20:54:37.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
192882025-11-12T20:54:37.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192892025-11-12T20:54:37.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192902025-11-12T20:54:37.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192912025-11-12T20:54:37.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192922025-11-12T20:54:37.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192932025-11-12T20:54:37.702ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
192942025-11-12T20:54:37.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192952025-11-12T20:54:37.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
192962025-11-12T20:54:37.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192972025-11-12T20:54:37.706ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
192982025-11-12T20:54:37.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
192992025-11-12T20:54:37.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193002025-11-12T20:54:37.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193012025-11-12T20:54:37.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193022025-11-12T20:54:37.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193032025-11-12T20:54:37.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
193042025-11-12T20:54:37.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193052025-11-12T20:54:37.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193062025-11-12T20:54:37.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193072025-11-12T20:54:37.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193082025-11-12T20:54:37.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193092025-11-12T20:54:37.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
193102025-11-12T20:54:37.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193112025-11-12T20:54:37.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193122025-11-12T20:54:37.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193132025-11-12T20:54:37.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193142025-11-12T20:54:37.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193152025-11-12T20:54:37.725ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
193162025-11-12T20:54:37.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193172025-11-12T20:54:37.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193182025-11-12T20:54:37.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193192025-11-12T20:54:37.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193202025-11-12T20:54:37.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193212025-11-12T20:54:37.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193222025-11-12T20:54:37.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193232025-11-12T20:54:37.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
193242025-11-12T20:54:37.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193252025-11-12T20:54:37.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193262025-11-12T20:54:37.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193272025-11-12T20:54:37.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
193282025-11-12T20:54:37.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193292025-11-12T20:54:37.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193302025-11-12T20:54:37.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193312025-11-12T20:54:37.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193322025-11-12T20:54:37.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193332025-11-12T20:54:37.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193342025-11-12T20:54:37.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193352025-11-12T20:54:37.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
193362025-11-12T20:54:37.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193372025-11-12T20:54:37.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193382025-11-12T20:54:37.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193392025-11-12T20:54:37.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
193402025-11-12T20:54:37.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193412025-11-12T20:54:37.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193422025-11-12T20:54:37.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193432025-11-12T20:54:37.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193442025-11-12T20:54:37.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193452025-11-12T20:54:37.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193462025-11-12T20:54:37.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193472025-11-12T20:54:37.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
193482025-11-12T20:54:37.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193492025-11-12T20:54:37.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193502025-11-12T20:54:37.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193512025-11-12T20:54:37.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193522025-11-12T20:54:37.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193532025-11-12T20:54:37.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
193542025-11-12T20:54:37.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193552025-11-12T20:54:37.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193562025-11-12T20:54:37.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193572025-11-12T20:54:37.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193582025-11-12T20:54:37.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193592025-11-12T20:54:37.790ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
193602025-11-12T20:54:37.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193612025-11-12T20:54:37.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193622025-11-12T20:54:37.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193632025-11-12T20:54:37.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193642025-11-12T20:54:37.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193652025-11-12T20:54:37.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
193662025-11-12T20:54:37.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193672025-11-12T20:54:37.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193682025-11-12T20:54:37.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193692025-11-12T20:54:37.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193702025-11-12T20:54:37.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193712025-11-12T20:54:37.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
193722025-11-12T20:54:37.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193732025-11-12T20:54:37.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193742025-11-12T20:54:37.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193752025-11-12T20:54:37.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193762025-11-12T20:54:37.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193772025-11-12T20:54:37.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
193782025-11-12T20:54:37.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193792025-11-12T20:54:37.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193802025-11-12T20:54:37.847ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
193812025-11-12T20:54:37.847ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
193822025-11-12T20:54:37.847ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 6e7a4dac-bf3e-4e68-9465-2cc23f334455 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
193832025-11-12T20:54:37.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193842025-11-12T20:54:37.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193852025-11-12T20:54:37.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193862025-11-12T20:54:37.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
193872025-11-12T20:54:37.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
193882025-11-12T20:54:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
193892025-11-12T20:54:37.855ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
193902025-11-12T20:54:37.855ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
193912025-11-12T20:54:37.855ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 231.964462ms iteration = 24 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
193922025-11-12T20:54:37.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 289 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = ab4b8310-bb81-44ee-a41d-f1a84cf4cc06 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
193932025-11-12T20:54:40.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 110 reason = Timeout
193942025-11-12T20:54:40.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193952025-11-12T20:54:40.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193962025-11-12T20:54:40.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193972025-11-12T20:54:40.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
193982025-11-12T20:54:40.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193992025-11-12T20:54:40.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194002025-11-12T20:54:40.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
194012025-11-12T20:54:40.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.413992ms iteration = 110 status = {"config_updated":false}
194022025-11-12T20:54:45.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
194032025-11-12T20:54:45.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
194042025-11-12T20:54:45.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194052025-11-12T20:54:45.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
194062025-11-12T20:54:45.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 111 reason = Timeout
194072025-11-12T20:54:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194082025-11-12T20:54:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
194092025-11-12T20:54:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194102025-11-12T20:54:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194112025-11-12T20:54:45.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194122025-11-12T20:54:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194132025-11-12T20:54:45.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
194142025-11-12T20:54:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
194152025-11-12T20:54:45.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.066696ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
194162025-11-12T20:54:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
194172025-11-12T20:54:45.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194182025-11-12T20:54:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
194192025-11-12T20:54:45.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194202025-11-12T20:54:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194212025-11-12T20:54:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194222025-11-12T20:54:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194232025-11-12T20:54:45.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
194242025-11-12T20:54:45.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.318735ms iteration = 111 status = {"config_updated":false}
194252025-11-12T20:54:45.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194262025-11-12T20:54:45.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194272025-11-12T20:54:45.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
194282025-11-12T20:54:45.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194292025-11-12T20:54:45.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194302025-11-12T20:54:45.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
194312025-11-12T20:54:45.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.778634ms iteration = 56 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
194322025-11-12T20:54:47.915ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = aeda0e96-ea3e-46ec-aa11-bb1756f27c80 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
194332025-11-12T20:54:47.915ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = aeda0e96-ea3e-46ec-aa11-bb1756f27c80 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
194342025-11-12T20:54:47.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = aeda0e96-ea3e-46ec-aa11-bb1756f27c80 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
194352025-11-12T20:54:47.918ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3403 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = aeda0e96-ea3e-46ec-aa11-bb1756f27c80 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
194362025-11-12T20:54:47.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 80de99c2-04b1-4c01-a20a-29787d63c438 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
194372025-11-12T20:54:47.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 80de99c2-04b1-4c01-a20a-29787d63c438 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
194382025-11-12T20:54:47.919ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 80de99c2-04b1-4c01-a20a-29787d63c438 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
194392025-11-12T20:54:47.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2821 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 80de99c2-04b1-4c01-a20a-29787d63c438 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
194402025-11-12T20:54:47.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 283 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = f392d92b-439a-4b6e-8bdb-b5ce429e9e3a response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
194412025-11-12T20:54:50.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 49 reason = Timeout
194422025-11-12T20:54:50.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
194432025-11-12T20:54:50.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
194442025-11-12T20:54:50.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 112 reason = Timeout
194452025-11-12T20:54:50.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194462025-11-12T20:54:50.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194472025-11-12T20:54:50.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194482025-11-12T20:54:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194492025-11-12T20:54:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
194502025-11-12T20:54:50.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
194512025-11-12T20:54:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
194522025-11-12T20:54:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
194532025-11-12T20:54:50.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194542025-11-12T20:54:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194552025-11-12T20:54:50.284ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da575366-ca7a-460e-9ae6-b9457f2d2443 original_time_started = 2025-11-12 20:54:36.970966 UTC
194562025-11-12T20:54:50.284ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 6.817847ms iteration = 49 status = {"Loaded":{"collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","time_loaded":"2025-11-12T20:54:50.277502783Z","time_started":"2025-11-12T20:54:36.970966Z"}}
194572025-11-12T20:54:50.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
194582025-11-12T20:54:50.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.408945ms iteration = 112 status = {"config_updated":false}
194592025-11-12T20:54:50.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 38 reason = Timeout
194602025-11-12T20:54:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
194612025-11-12T20:54:50.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
194622025-11-12T20:54:50.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194632025-11-12T20:54:50.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
194642025-11-12T20:54:50.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
194652025-11-12T20:54:50.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
194662025-11-12T20:54:50.290ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.01453ms iteration = 38 status = "NoSitrep"
194672025-11-12T20:54:50.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 38 reason = Timeout
194682025-11-12T20:54:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
194692025-11-12T20:54:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
194702025-11-12T20:54:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
194712025-11-12T20:54:50.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
194722025-11-12T20:54:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194732025-11-12T20:54:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
194742025-11-12T20:54:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
194752025-11-12T20:54:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
194762025-11-12T20:54:50.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
194772025-11-12T20:54:50.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
194782025-11-12T20:54:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194792025-11-12T20:54:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
194802025-11-12T20:54:50.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
194812025-11-12T20:54:50.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.697562ms iteration = 38 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
194822025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
194832025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
194842025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
194852025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 113 reason = Timeout
194862025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194872025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194882025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
194892025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194902025-11-12T20:54:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194912025-11-12T20:54:55.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194922025-11-12T20:54:55.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
194932025-11-12T20:54:55.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194942025-11-12T20:54:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
194952025-11-12T20:54:55.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.967122ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
194962025-11-12T20:54:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
194972025-11-12T20:54:55.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194982025-11-12T20:54:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
194992025-11-12T20:54:55.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195002025-11-12T20:54:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195012025-11-12T20:54:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195022025-11-12T20:54:55.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
195032025-11-12T20:54:55.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.402531ms iteration = 113 status = {"config_updated":false}
195042025-11-12T20:54:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195052025-11-12T20:54:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195062025-11-12T20:54:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195072025-11-12T20:54:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
195082025-11-12T20:54:55.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195092025-11-12T20:54:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195102025-11-12T20:54:55.301ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
195112025-11-12T20:54:55.302ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.461448ms iteration = 57 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
195122025-11-12T20:54:57.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 345 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 85cf4164-e30d-4cad-8298-664d7bb1de33 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
195132025-11-12T20:55:00.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 114 reason = Timeout
195142025-11-12T20:55:00.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195152025-11-12T20:55:00.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195162025-11-12T20:55:00.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195172025-11-12T20:55:00.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
195182025-11-12T20:55:00.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195192025-11-12T20:55:00.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195202025-11-12T20:55:00.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
195212025-11-12T20:55:00.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.312714ms iteration = 114 status = {"config_updated":false}
195222025-11-12T20:55:05.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
195232025-11-12T20:55:05.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
195242025-11-12T20:55:05.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
195252025-11-12T20:55:05.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
195262025-11-12T20:55:05.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
195272025-11-12T20:55:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
195282025-11-12T20:55:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
195292025-11-12T20:55:05.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 50 reason = Timeout
195302025-11-12T20:55:05.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
195312025-11-12T20:55:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
195322025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
195332025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
195342025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
195352025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
195362025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
195372025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
195382025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
195392025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
195402025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
195412025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
195422025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
195432025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195442025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
195452025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
195462025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
195472025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195482025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
195492025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
195502025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
195512025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
195522025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
195532025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
195542025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
195552025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
195562025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
195572025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
195582025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
195592025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
195602025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
195612025-11-12T20:55:05.278ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
195622025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
195632025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
195642025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
195652025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
195662025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
195672025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 115 reason = Timeout
195682025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195692025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
195702025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
195712025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
195722025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
195732025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195742025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
195752025-11-12T20:55:05.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1500, tv_nsec: 775336641 } })
195762025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195772025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
195782025-11-12T20:55:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
195792025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
195802025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
195812025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
195822025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195832025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
195842025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
195852025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
195862025-11-12T20:55:05.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
195872025-11-12T20:55:05.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1500, tv_nsec: 775620148 } })
195882025-11-12T20:55:05.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
195892025-11-12T20:55:05.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1500, tv_nsec: 776086590 } })
195902025-11-12T20:55:05.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195912025-11-12T20:55:05.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195922025-11-12T20:55:05.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195932025-11-12T20:55:05.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195942025-11-12T20:55:05.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195952025-11-12T20:55:05.283ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
195962025-11-12T20:55:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
195972025-11-12T20:55:05.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 39 reason = Timeout
195982025-11-12T20:55:05.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
195992025-11-12T20:55:05.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196002025-11-12T20:55:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
196012025-11-12T20:55:05.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196022025-11-12T20:55:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
196032025-11-12T20:55:05.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
196042025-11-12T20:55:05.286ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196052025-11-12T20:55:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
196062025-11-12T20:55:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
196072025-11-12T20:55:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
196082025-11-12T20:55:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
196092025-11-12T20:55:05.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
196102025-11-12T20:55:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
196112025-11-12T20:55:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
196122025-11-12T20:55:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
196132025-11-12T20:55:05.289ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
196142025-11-12T20:55:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
196152025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196162025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
196172025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
196182025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
196192025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
196202025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
196212025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 14.813453ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
196222025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
196232025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
196242025-11-12T20:55:05.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
196252025-11-12T20:55:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
196262025-11-12T20:55:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
196272025-11-12T20:55:05.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 39 reason = Timeout
196282025-11-12T20:55:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196292025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
196302025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
196312025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196322025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
196332025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 13.705434ms iteration = 115 status = {"config_updated":false}
196342025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
196352025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
196362025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196372025-11-12T20:55:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196382025-11-12T20:55:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196392025-11-12T20:55:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196402025-11-12T20:55:05.293ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
196412025-11-12T20:55:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
196422025-11-12T20:55:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
196432025-11-12T20:55:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 16.384944ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
196442025-11-12T20:55:05.297ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196452025-11-12T20:55:05.298ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1500, tv_nsec: 794520387 } })
196462025-11-12T20:55:05.298ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
196472025-11-12T20:55:05.298ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 14.113865ms iteration = 39 status = "NoSitrep"
196482025-11-12T20:55:05.301ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da575366-ca7a-460e-9ae6-b9457f2d2443 original_time_started = 2025-11-12 20:54:36.970966 UTC
196492025-11-12T20:55:05.301ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 25.021278ms iteration = 50 status = {"Loaded":{"collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","time_loaded":"2025-11-12T20:55:05.276819631Z","time_started":"2025-11-12T20:54:36.970966Z"}}
196502025-11-12T20:55:05.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196512025-11-12T20:55:05.302ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196522025-11-12T20:55:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
196532025-11-12T20:55:05.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
196542025-11-12T20:55:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
196552025-11-12T20:55:05.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
196562025-11-12T20:55:05.306ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
196572025-11-12T20:55:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
196582025-11-12T20:55:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
196592025-11-12T20:55:05.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
196602025-11-12T20:55:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
196612025-11-12T20:55:05.313ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 35.443207ms iteration = 20 status = 0
196622025-11-12T20:55:05.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196632025-11-12T20:55:05.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196642025-11-12T20:55:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196652025-11-12T20:55:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
196662025-11-12T20:55:05.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196672025-11-12T20:55:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196682025-11-12T20:55:05.332ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 54.806629ms iteration = 32 status = {"physical_disks_added":0}
196692025-11-12T20:55:05.357ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
196702025-11-12T20:55:05.357ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 79.480827ms iteration = 58 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
196712025-11-12T20:55:05.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.181661ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
196722025-11-12T20:55:05.360ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
196732025-11-12T20:55:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
196742025-11-12T20:55:05.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.46793ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
196752025-11-12T20:55:05.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
196762025-11-12T20:55:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
196772025-11-12T20:55:05.365ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
196782025-11-12T20:55:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
196792025-11-12T20:55:05.366ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196802025-11-12T20:55:05.368ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 89.896146ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
196812025-11-12T20:55:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
196822025-11-12T20:55:05.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
196832025-11-12T20:55:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
196842025-11-12T20:55:05.372ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
196852025-11-12T20:55:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
196862025-11-12T20:55:05.373ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
196872025-11-12T20:55:05.373ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
196882025-11-12T20:55:05.373ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
196892025-11-12T20:55:05.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
196902025-11-12T20:55:05.374ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
196912025-11-12T20:55:05.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
196922025-11-12T20:55:05.377ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196932025-11-12T20:55:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
196942025-11-12T20:55:05.387ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
196952025-11-12T20:55:05.387ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 108.853997ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
196962025-11-12T20:55:05.390ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196972025-11-12T20:55:05.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196982025-11-12T20:55:05.391ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196992025-11-12T20:55:05.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
197002025-11-12T20:55:05.393ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197012025-11-12T20:55:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
197022025-11-12T20:55:05.396ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
197032025-11-12T20:55:05.396ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 105.163471ms iteration = 39 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
197042025-11-12T20:55:05.415ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
197052025-11-12T20:55:05.415ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
197062025-11-12T20:55:05.416ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f95c59c5-7863-4f0c-ae09-643ede867c0b", "content-length": "26", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197072025-11-12T20:55:05.416ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
197082025-11-12T20:55:05.416ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 138.550835ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
197092025-11-12T20:55:05.417ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 139.644954ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
197102025-11-12T20:55:05.419ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
197112025-11-12T20:55:05.426ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
197122025-11-12T20:55:05.426ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd8defd6-d8d3-4ab0-a62f-bef605180b7a", "content-length": "78", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197132025-11-12T20:55:05.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 148.443262ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
197142025-11-12T20:55:05.426ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
197152025-11-12T20:55:05.426ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
197162025-11-12T20:55:05.426ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
197172025-11-12T20:55:05.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
197182025-11-12T20:55:05.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
197192025-11-12T20:55:05.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
197202025-11-12T20:55:05.428ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
197212025-11-12T20:55:05.429ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
197222025-11-12T20:55:05.429ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
197232025-11-12T20:55:05.432ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
197242025-11-12T20:55:05.432ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
197252025-11-12T20:55:05.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
197262025-11-12T20:55:05.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
197272025-11-12T20:55:05.433ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
197282025-11-12T20:55:05.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
197292025-11-12T20:55:05.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
197302025-11-12T20:55:05.433ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
197312025-11-12T20:55:05.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
197322025-11-12T20:55:05.434ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
197332025-11-12T20:55:05.434ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
197342025-11-12T20:55:05.435ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
197352025-11-12T20:55:05.435ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
197362025-11-12T20:55:05.435ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "81e1a2dc-9604-4ebf-b9d7-47400c715c31", "content-length": "81", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197372025-11-12T20:55:05.435ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "972f7c4e-889d-48bb-b8f6-8be903a83d56", "content-length": "81", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197382025-11-12T20:55:05.436ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
197392025-11-12T20:55:05.436ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
197402025-11-12T20:55:05.436ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
197412025-11-12T20:55:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
197422025-11-12T20:55:05.436ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
197432025-11-12T20:55:05.436ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 158.140183ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
197442025-11-12T20:55:05.444ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
197452025-11-12T20:55:05.446ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 170.866041ms iteration = 20 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
197462025-11-12T20:55:05.448ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
197472025-11-12T20:55:05.448ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
197482025-11-12T20:55:05.449ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "491cdab7-7c3e-42b7-9ad7-60932570e7f5", "content-length": "26", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197492025-11-12T20:55:05.449ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
197502025-11-12T20:55:05.450ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
197512025-11-12T20:55:05.451ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
197522025-11-12T20:55:05.451ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
197532025-11-12T20:55:05.455ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e86ae0b3-b442-4346-bfa0-2feec65bd00b", "content-length": "372", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197542025-11-12T20:55:05.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
197552025-11-12T20:55:05.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
197562025-11-12T20:55:05.469ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
197572025-11-12T20:55:05.469ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
197582025-11-12T20:55:05.472ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7779b76c-5078-4899-8893-3ff7195e2840", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197592025-11-12T20:55:05.472ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
197602025-11-12T20:55:05.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
197612025-11-12T20:55:05.475ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
197622025-11-12T20:55:05.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
197632025-11-12T20:55:05.479ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
197642025-11-12T20:55:05.479ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
197652025-11-12T20:55:05.479ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 200.765056ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
197662025-11-12T20:55:05.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
197672025-11-12T20:55:05.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
197682025-11-12T20:55:05.480ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
197692025-11-12T20:55:05.480ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
197702025-11-12T20:55:05.480ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a1dbae4-21ac-4627-a9a5-37aac84879d2", "content-length": "81", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197712025-11-12T20:55:05.482ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 204.198614ms iteration = 20 status = {"disabled":false,"errors":[],"sps":[]}
197722025-11-12T20:55:05.482ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
197732025-11-12T20:55:05.482ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
197742025-11-12T20:55:05.482ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
197752025-11-12T20:55:05.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
197762025-11-12T20:55:05.488ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "26a3105d-b624-4932-a62c-1d6aaa910d34", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197772025-11-12T20:55:05.490ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 211.982546ms iteration = 30 status = {}
197782025-11-12T20:55:05.490ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
197792025-11-12T20:55:05.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
197802025-11-12T20:55:05.492ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197812025-11-12T20:55:05.494ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
197822025-11-12T20:55:05.494ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
197832025-11-12T20:55:05.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
197842025-11-12T20:55:05.496ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197852025-11-12T20:55:05.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
197862025-11-12T20:55:05.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
197872025-11-12T20:55:05.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
197882025-11-12T20:55:05.512ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 237.066184ms iteration = 20 status = {}
197892025-11-12T20:55:05.516ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 13385411 } })
197902025-11-12T20:55:05.565ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
197912025-11-12T20:55:05.566ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1be46302-f89d-4d00-8188-498a14043b3e", "content-length": "691", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
197922025-11-12T20:55:05.566ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
197932025-11-12T20:55:05.566ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
197942025-11-12T20:55:05.566ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
197952025-11-12T20:55:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 290.518278ms iteration = 20 status = {}
197962025-11-12T20:55:05.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
197972025-11-12T20:55:05.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
197982025-11-12T20:55:05.608ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
197992025-11-12T20:55:05.608ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198002025-11-12T20:55:05.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198012025-11-12T20:55:05.618ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
198022025-11-12T20:55:05.618ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
198032025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
198042025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
198052025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
198062025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
198072025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
198082025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
198092025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
198102025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
198112025-11-12T20:55:05.619ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
198122025-11-12T20:55:05.619ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198132025-11-12T20:55:05.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198142025-11-12T20:55:05.622ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
198152025-11-12T20:55:05.622ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
198162025-11-12T20:55:05.729ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8f6cafec-0420-4b7b-ac54-848036372867", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
198172025-11-12T20:55:05.730ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
198182025-11-12T20:55:05.730ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
198192025-11-12T20:55:05.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "43c57bba-4103-42c2-bb4f-b20cd7534a6d", "content-length": "682", "date": "Wed, 12 Nov 2025 20:55:05 GMT"} })
198202025-11-12T20:55:05.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198212025-11-12T20:55:05.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198222025-11-12T20:55:05.734ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198232025-11-12T20:55:05.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198242025-11-12T20:55:05.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198252025-11-12T20:55:05.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198262025-11-12T20:55:05.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198272025-11-12T20:55:05.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198282025-11-12T20:55:05.747ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 469.50068ms iteration = 20 status = {}
198292025-11-12T20:55:07.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 299 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 8dfe4c0a-a283-4c6b-aa27-51005ce556a0 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
198302025-11-12T20:55:10.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 116 reason = Timeout
198312025-11-12T20:55:10.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198322025-11-12T20:55:10.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198332025-11-12T20:55:10.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198342025-11-12T20:55:10.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
198352025-11-12T20:55:10.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198362025-11-12T20:55:10.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198372025-11-12T20:55:10.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
198382025-11-12T20:55:10.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.132615ms iteration = 116 status = {"config_updated":false}
198392025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
198402025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
198412025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
198422025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
198432025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 117 reason = Timeout
198442025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198452025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
198462025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
198472025-11-12T20:55:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198482025-11-12T20:55:15.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198492025-11-12T20:55:15.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
198502025-11-12T20:55:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
198512025-11-12T20:55:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198522025-11-12T20:55:15.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.773457ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
198532025-11-12T20:55:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
198542025-11-12T20:55:15.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
198552025-11-12T20:55:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
198562025-11-12T20:55:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
198572025-11-12T20:55:15.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198582025-11-12T20:55:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198592025-11-12T20:55:15.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
198602025-11-12T20:55:15.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
198612025-11-12T20:55:15.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
198622025-11-12T20:55:15.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.968036ms iteration = 117 status = {"config_updated":false}
198632025-11-12T20:55:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198642025-11-12T20:55:15.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
198652025-11-12T20:55:15.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
198662025-11-12T20:55:15.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
198672025-11-12T20:55:15.305ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
198682025-11-12T20:55:15.305ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.42854ms iteration = 59 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
198692025-11-12T20:55:17.915ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 5026f583-01c4-444d-9d4d-da5b3ecdf00c uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
198702025-11-12T20:55:17.915ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 5026f583-01c4-444d-9d4d-da5b3ecdf00c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
198712025-11-12T20:55:17.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 5026f583-01c4-444d-9d4d-da5b3ecdf00c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
198722025-11-12T20:55:17.918ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3009 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = 5026f583-01c4-444d-9d4d-da5b3ecdf00c response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
198732025-11-12T20:55:17.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = c68f1d29-8481-4a78-be9e-0fe6ae65491c uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
198742025-11-12T20:55:17.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = c68f1d29-8481-4a78-be9e-0fe6ae65491c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
198752025-11-12T20:55:17.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = c68f1d29-8481-4a78-be9e-0fe6ae65491c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
198762025-11-12T20:55:17.921ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2647 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34179 req_id = c68f1d29-8481-4a78-be9e-0fe6ae65491c response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
198772025-11-12T20:55:17.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 267 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = d47d95b8-c17d-4a1c-83a8-89f88340dfb6 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
198782025-11-12T20:55:20.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 51 reason = Timeout
198792025-11-12T20:55:20.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
198802025-11-12T20:55:20.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
198812025-11-12T20:55:20.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 118 reason = Timeout
198822025-11-12T20:55:20.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198832025-11-12T20:55:20.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198842025-11-12T20:55:20.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198852025-11-12T20:55:20.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198862025-11-12T20:55:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
198872025-11-12T20:55:20.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
198882025-11-12T20:55:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
198892025-11-12T20:55:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
198902025-11-12T20:55:20.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198912025-11-12T20:55:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198922025-11-12T20:55:20.284ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da575366-ca7a-460e-9ae6-b9457f2d2443 original_time_started = 2025-11-12 20:54:36.970966 UTC
198932025-11-12T20:55:20.284ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.666658ms iteration = 51 status = {"Loaded":{"collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","time_loaded":"2025-11-12T20:55:20.276602084Z","time_started":"2025-11-12T20:54:36.970966Z"}}
198942025-11-12T20:55:20.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 40 reason = Timeout
198952025-11-12T20:55:20.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
198962025-11-12T20:55:20.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
198972025-11-12T20:55:20.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
198982025-11-12T20:55:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.489094ms iteration = 118 status = {"config_updated":false}
198992025-11-12T20:55:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199002025-11-12T20:55:20.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
199012025-11-12T20:55:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
199022025-11-12T20:55:20.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
199032025-11-12T20:55:20.290ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.901032ms iteration = 40 status = "NoSitrep"
199042025-11-12T20:55:20.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 40 reason = Timeout
199052025-11-12T20:55:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
199062025-11-12T20:55:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
199072025-11-12T20:55:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
199082025-11-12T20:55:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
199092025-11-12T20:55:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199102025-11-12T20:55:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
199112025-11-12T20:55:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
199122025-11-12T20:55:20.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
199132025-11-12T20:55:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
199142025-11-12T20:55:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
199152025-11-12T20:55:20.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199162025-11-12T20:55:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
199172025-11-12T20:55:20.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
199182025-11-12T20:55:20.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.536242ms iteration = 40 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
199192025-11-12T20:55:25.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
199202025-11-12T20:55:25.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
199212025-11-12T20:55:25.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
199222025-11-12T20:55:25.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 119 reason = Timeout
199232025-11-12T20:55:25.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
199242025-11-12T20:55:25.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
199252025-11-12T20:55:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
199262025-11-12T20:55:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
199272025-11-12T20:55:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
199282025-11-12T20:55:25.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
199292025-11-12T20:55:25.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199302025-11-12T20:55:25.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
199312025-11-12T20:55:25.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199322025-11-12T20:55:25.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.987964ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
199332025-11-12T20:55:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
199342025-11-12T20:55:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
199352025-11-12T20:55:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
199362025-11-12T20:55:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
199372025-11-12T20:55:25.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
199382025-11-12T20:55:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
199392025-11-12T20:55:25.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
199402025-11-12T20:55:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
199412025-11-12T20:55:25.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
199422025-11-12T20:55:25.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.286064ms iteration = 119 status = {"config_updated":false}
199432025-11-12T20:55:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199442025-11-12T20:55:25.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
199452025-11-12T20:55:25.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
199462025-11-12T20:55:25.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
199472025-11-12T20:55:25.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
199482025-11-12T20:55:25.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.764657ms iteration = 60 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
199492025-11-12T20:55:27.938ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 409 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 534f7fc0-f98d-45ca-89ec-0764ee284830 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
199502025-11-12T20:55:30.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 120 reason = Timeout
199512025-11-12T20:55:30.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
199522025-11-12T20:55:30.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
199532025-11-12T20:55:30.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199542025-11-12T20:55:30.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
199552025-11-12T20:55:30.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
199562025-11-12T20:55:30.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
199572025-11-12T20:55:30.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
199582025-11-12T20:55:30.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.802696ms iteration = 120 status = {"config_updated":false}
199592025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
199602025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
199612025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
199622025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
199632025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
199642025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
199652025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
199662025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
199672025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
199682025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
199692025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
199702025-11-12T20:55:35.275ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
199712025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
199722025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
199732025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
199742025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
199752025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
199762025-11-12T20:55:35.275ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = ExternalNetworking step id = Ensure
199772025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
199782025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
199792025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
199802025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
199812025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
199822025-11-12T20:55:35.275ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:45:35.275453131 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
199832025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
199842025-11-12T20:55:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1530, tv_nsec: 772605908 } })
199852025-11-12T20:55:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1530, tv_nsec: 772625908 } })
199862025-11-12T20:55:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
199872025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
199882025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
199892025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
199902025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
199912025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
199922025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
199932025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Timeout
199942025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
199952025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 52 reason = Timeout
199962025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
199972025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
199982025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.470778ms iteration = 11 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
199992025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
200002025-11-12T20:55:35.276ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.785556ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
200012025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
200022025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
200032025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
200042025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
200052025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
200062025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
200072025-11-12T20:55:35.277ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
200082025-11-12T20:55:35.277ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
200092025-11-12T20:55:35.277ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
200102025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
200112025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
200122025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
200132025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
200142025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
200152025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
200162025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
200172025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
200182025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
200192025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
200202025-11-12T20:55:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
200212025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
200222025-11-12T20:55:35.278ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
200232025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
200242025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
200252025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 121 reason = Timeout
200262025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
200272025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
200282025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
200292025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
200302025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
200312025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
200322025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
200332025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 25 reason = Timeout
200342025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
200352025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
200362025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
200372025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
200382025-11-12T20:55:35.278ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
200392025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
200402025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
200412025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
200422025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
200432025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
200442025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
200452025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
200462025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 3 reason = Timeout
200472025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
200482025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
200492025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
200502025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
200512025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
200522025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
200532025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
200542025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
200552025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
200562025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
200572025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
200582025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
200592025-11-12T20:55:35.278ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
200602025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
200612025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
200622025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
200632025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 15 reason = Timeout
200642025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
200652025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
200662025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
200672025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
200682025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
200692025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
200702025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
200712025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
200722025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
200732025-11-12T20:55:35.278ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
200742025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
200752025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
200762025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
200772025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
200782025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
200792025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
200802025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
200812025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
200822025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
200832025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
200842025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
200852025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
200862025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
200872025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
200882025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
200892025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
200902025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
200912025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
200922025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
200932025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
200942025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
200952025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
200962025-11-12T20:55:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
200972025-11-12T20:55:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
200982025-11-12T20:55:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
200992025-11-12T20:55:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1530, tv_nsec: 775729169 } })
201002025-11-12T20:55:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
201012025-11-12T20:55:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
201022025-11-12T20:55:35.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201032025-11-12T20:55:35.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201042025-11-12T20:55:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201052025-11-12T20:55:35.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201062025-11-12T20:55:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201072025-11-12T20:55:35.283ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1530, tv_nsec: 779562038 } })
201082025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
201092025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
201102025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 41 reason = Timeout
201112025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
201122025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
201132025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
201142025-11-12T20:55:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.522852ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
201152025-11-12T20:55:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
201162025-11-12T20:55:35.286ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
201172025-11-12T20:55:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
201182025-11-12T20:55:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
201192025-11-12T20:55:35.289ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201202025-11-12T20:55:35.289ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
201212025-11-12T20:55:35.289ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 11.20943ms iteration = 11 status = {"by_rx":{},"error":null}
201222025-11-12T20:55:35.289ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.368214ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
201232025-11-12T20:55:35.289ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:45:35.275453131 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
201242025-11-12T20:55:35.289ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.067604ms iteration = 11 status = {"expiration":"2025-11-12T20:45:35.275453131Z","pruned":[]}
201252025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201262025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
201272025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 11 reason = Timeout
201282025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 41 reason = Timeout
201292025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
201302025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
201312025-11-12T20:55:35.290ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
201322025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
201332025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
201342025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
201352025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
201362025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
201372025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
201382025-11-12T20:55:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
201392025-11-12T20:55:35.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201402025-11-12T20:55:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
201412025-11-12T20:55:35.291ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1530, tv_nsec: 788475039 } })
201422025-11-12T20:55:35.292ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201432025-11-12T20:55:35.292ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
201442025-11-12T20:55:35.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201452025-11-12T20:55:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
201462025-11-12T20:55:35.294ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
201472025-11-12T20:55:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
201482025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
201492025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
201502025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
201512025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201522025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
201532025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201542025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201552025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201562025-11-12T20:55:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
201572025-11-12T20:55:35.297ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201582025-11-12T20:55:35.297ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201592025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
201602025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
201612025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
201622025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
201632025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
201642025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
201652025-11-12T20:55:35.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
201662025-11-12T20:55:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
201672025-11-12T20:55:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
201682025-11-12T20:55:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
201692025-11-12T20:55:35.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
201702025-11-12T20:55:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
201712025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
201722025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
201732025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
201742025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201752025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
201762025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201772025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201782025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
201792025-11-12T20:55:35.302ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
201802025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
201812025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
201822025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
201832025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201842025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
201852025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
201862025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
201872025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
201882025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
201892025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201902025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201912025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
201922025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
201932025-11-12T20:55:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
201942025-11-12T20:55:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
201952025-11-12T20:55:35.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
201962025-11-12T20:55:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
201972025-11-12T20:55:35.305ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
201982025-11-12T20:55:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
201992025-11-12T20:55:35.306ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202002025-11-12T20:55:35.306ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
202012025-11-12T20:55:35.306ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 28.523008ms iteration = 121 status = {"config_updated":false}
202022025-11-12T20:55:35.306ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202032025-11-12T20:55:35.307ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202042025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
202052025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202062025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
202072025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
202082025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
202092025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
202102025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202112025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
202122025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
202132025-11-12T20:55:35.311ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 33.526698ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
202142025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
202152025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
202162025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
202172025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
202182025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202192025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 34.147014ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
202202025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
202212025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
202222025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202232025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da575366-ca7a-460e-9ae6-b9457f2d2443 original_time_started = 2025-11-12 20:54:36.970966 UTC
202242025-11-12T20:55:35.312ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 36.173246ms iteration = 52 status = {"Loaded":{"collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","time_loaded":"2025-11-12T20:55:35.276806016Z","time_started":"2025-11-12T20:54:36.970966Z"}}
202252025-11-12T20:55:35.313ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
202262025-11-12T20:55:35.313ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
202272025-11-12T20:55:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
202282025-11-12T20:55:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
202292025-11-12T20:55:35.314ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
202302025-11-12T20:55:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
202312025-11-12T20:55:35.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202322025-11-12T20:55:35.316ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
202332025-11-12T20:55:35.316ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
202342025-11-12T20:55:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
202352025-11-12T20:55:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
202362025-11-12T20:55:35.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202372025-11-12T20:55:35.318ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202382025-11-12T20:55:35.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
202392025-11-12T20:55:35.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
202402025-11-12T20:55:35.319ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
202412025-11-12T20:55:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
202422025-11-12T20:55:35.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202432025-11-12T20:55:35.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
202442025-11-12T20:55:35.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202452025-11-12T20:55:35.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
202462025-11-12T20:55:35.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202472025-11-12T20:55:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202482025-11-12T20:55:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
202492025-11-12T20:55:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
202502025-11-12T20:55:35.325ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
202512025-11-12T20:55:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
202522025-11-12T20:55:35.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
202532025-11-12T20:55:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
202542025-11-12T20:55:35.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
202552025-11-12T20:55:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
202562025-11-12T20:55:35.328ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
202572025-11-12T20:55:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
202582025-11-12T20:55:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
202592025-11-12T20:55:35.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
202602025-11-12T20:55:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
202612025-11-12T20:55:35.331ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 46.492284ms iteration = 41 status = "NoSitrep"
202622025-11-12T20:55:35.332ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
202632025-11-12T20:55:35.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
202642025-11-12T20:55:35.332ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1530, tv_nsec: 829369147 } })
202652025-11-12T20:55:35.333ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
202662025-11-12T20:55:35.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202672025-11-12T20:55:35.337ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202682025-11-12T20:55:35.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202692025-11-12T20:55:35.339ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
202702025-11-12T20:55:35.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
202712025-11-12T20:55:35.341ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202722025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
202732025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
202742025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
202752025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
202762025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
202772025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
202782025-11-12T20:55:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
202792025-11-12T20:55:35.345ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202802025-11-12T20:55:35.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
202812025-11-12T20:55:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
202822025-11-12T20:55:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
202832025-11-12T20:55:35.347ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
202842025-11-12T20:55:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
202852025-11-12T20:55:35.348ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
202862025-11-12T20:55:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
202872025-11-12T20:55:35.350ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
202882025-11-12T20:55:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
202892025-11-12T20:55:35.358ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 80.490583ms iteration = 21 status = 0
202902025-11-12T20:55:35.363ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 85.198475ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
202912025-11-12T20:55:35.364ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
202922025-11-12T20:55:35.366ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
202932025-11-12T20:55:35.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202942025-11-12T20:55:35.370ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202952025-11-12T20:55:35.371ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
202962025-11-12T20:55:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
202972025-11-12T20:55:35.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
202982025-11-12T20:55:35.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
202992025-11-12T20:55:35.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
203002025-11-12T20:55:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
203012025-11-12T20:55:35.374ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
203022025-11-12T20:55:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
203032025-11-12T20:55:35.374ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
203042025-11-12T20:55:35.377ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
203052025-11-12T20:55:35.383ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
203062025-11-12T20:55:35.383ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
203072025-11-12T20:55:35.383ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
203082025-11-12T20:55:35.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
203092025-11-12T20:55:35.384ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
203102025-11-12T20:55:35.385ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
203112025-11-12T20:55:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
203122025-11-12T20:55:35.389ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 111.012283ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
203132025-11-12T20:55:35.393ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
203142025-11-12T20:55:35.396ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
203152025-11-12T20:55:35.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203162025-11-12T20:55:35.399ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
203172025-11-12T20:55:35.401ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
203182025-11-12T20:55:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
203192025-11-12T20:55:35.402ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
203202025-11-12T20:55:35.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
203212025-11-12T20:55:35.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
203222025-11-12T20:55:35.403ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
203232025-11-12T20:55:35.404ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = Some(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint)) file = nexus/types/src/quiesce.rs:525 generation = 1
203242025-11-12T20:55:35.404ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 126.164265ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
203252025-11-12T20:55:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
203262025-11-12T20:55:35.406ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
203272025-11-12T20:55:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
203282025-11-12T20:55:35.407ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
203292025-11-12T20:55:35.407ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
203302025-11-12T20:55:35.407ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 129.189474ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
203312025-11-12T20:55:35.407ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203322025-11-12T20:55:35.407ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
203332025-11-12T20:55:35.407ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 129.495831ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
203342025-11-12T20:55:35.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 131.672498ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
203352025-11-12T20:55:35.410ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
203362025-11-12T20:55:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 133.34298ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
203372025-11-12T20:55:35.411ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
203382025-11-12T20:55:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 121.269339ms iteration = 41 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
203392025-11-12T20:55:35.416ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
203402025-11-12T20:55:35.416ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
203412025-11-12T20:55:35.416ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
203422025-11-12T20:55:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
203432025-11-12T20:55:35.417ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
203442025-11-12T20:55:35.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203452025-11-12T20:55:35.419ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
203462025-11-12T20:55:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
203472025-11-12T20:55:35.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
203482025-11-12T20:55:35.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
203492025-11-12T20:55:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
203502025-11-12T20:55:35.423ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
203512025-11-12T20:55:35.423ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
203522025-11-12T20:55:35.423ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 145.056503ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
203532025-11-12T20:55:35.423ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
203542025-11-12T20:55:35.424ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
203552025-11-12T20:55:35.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
203562025-11-12T20:55:35.424ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203572025-11-12T20:55:35.424ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = SledList step id = Fetch
203582025-11-12T20:55:35.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203592025-11-12T20:55:35.425ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
203602025-11-12T20:55:35.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
203612025-11-12T20:55:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203622025-11-12T20:55:35.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
203632025-11-12T20:55:35.426ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203642025-11-12T20:55:35.426ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
203652025-11-12T20:55:35.426ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
203662025-11-12T20:55:35.426ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
203672025-11-12T20:55:35.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
203682025-11-12T20:55:35.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203692025-11-12T20:55:35.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203702025-11-12T20:55:35.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203712025-11-12T20:55:35.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
203722025-11-12T20:55:35.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203732025-11-12T20:55:35.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
203742025-11-12T20:55:35.433ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
203752025-11-12T20:55:35.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
203762025-11-12T20:55:35.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203772025-11-12T20:55:35.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
203782025-11-12T20:55:35.434ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
203792025-11-12T20:55:35.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
203802025-11-12T20:55:35.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
203812025-11-12T20:55:35.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
203822025-11-12T20:55:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
203832025-11-12T20:55:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
203842025-11-12T20:55:35.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
203852025-11-12T20:55:35.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
203862025-11-12T20:55:35.438ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:114 status = TufRepoPrunerStatus { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
203872025-11-12T20:55:35.438ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 160.627337ms iteration = 3 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"warnings":[]}
203882025-11-12T20:55:35.439ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
203892025-11-12T20:55:35.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
203902025-11-12T20:55:35.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203912025-11-12T20:55:35.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203922025-11-12T20:55:35.441ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
203932025-11-12T20:55:35.441ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
203942025-11-12T20:55:35.441ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
203952025-11-12T20:55:35.441ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
203962025-11-12T20:55:35.441ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 165.966285ms iteration = 11 status = {"generation":1}
203972025-11-12T20:55:35.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
203982025-11-12T20:55:35.443ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 164.766904ms iteration = 33 status = {"physical_disks_added":0}
203992025-11-12T20:55:35.443ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204002025-11-12T20:55:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204012025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
204022025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204032025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
204042025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
204052025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204062025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
204072025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
204082025-11-12T20:55:35.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
204092025-11-12T20:55:35.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
204102025-11-12T20:55:35.445ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 166.9395ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
204112025-11-12T20:55:35.445ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
204122025-11-12T20:55:35.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
204132025-11-12T20:55:35.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
204142025-11-12T20:55:35.445ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204152025-11-12T20:55:35.447ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204162025-11-12T20:55:35.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204172025-11-12T20:55:35.447ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
204182025-11-12T20:55:35.447ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 169.449445ms iteration = 15 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"da575366-ca7a-460e-9ae6-b9457f2d2443","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
204192025-11-12T20:55:35.448ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
204202025-11-12T20:55:35.448ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 170.139503ms iteration = 61 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
204212025-11-12T20:55:35.448ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = DeployNexusRecords step id = Ensure
204222025-11-12T20:55:35.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204232025-11-12T20:55:35.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204242025-11-12T20:55:35.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204252025-11-12T20:55:35.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
204262025-11-12T20:55:35.453ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
204272025-11-12T20:55:35.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
204282025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
204292025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204302025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204312025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
204322025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
204332025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
204342025-11-12T20:55:35.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204352025-11-12T20:55:35.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204362025-11-12T20:55:35.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
204372025-11-12T20:55:35.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204382025-11-12T20:55:35.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204392025-11-12T20:55:35.462ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
204402025-11-12T20:55:35.462ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
204412025-11-12T20:55:35.462ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
204422025-11-12T20:55:35.462ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 187.43993ms iteration = 11 status = {"generation":2}
204432025-11-12T20:55:35.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204442025-11-12T20:55:35.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
204452025-11-12T20:55:35.469ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204462025-11-12T20:55:35.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204472025-11-12T20:55:35.473ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204482025-11-12T20:55:35.473ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = SledAgent step id = Ensure
204492025-11-12T20:55:35.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204502025-11-12T20:55:35.477ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204512025-11-12T20:55:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204522025-11-12T20:55:35.486ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204532025-11-12T20:55:35.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204542025-11-12T20:55:35.492ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 214.594923ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
204552025-11-12T20:55:35.493ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204562025-11-12T20:55:35.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204572025-11-12T20:55:35.498ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204582025-11-12T20:55:35.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204592025-11-12T20:55:35.501ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204602025-11-12T20:55:35.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
204612025-11-12T20:55:35.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
204622025-11-12T20:55:35.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204632025-11-12T20:55:35.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
204642025-11-12T20:55:35.508ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204652025-11-12T20:55:35.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204662025-11-12T20:55:35.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
204672025-11-12T20:55:35.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
204682025-11-12T20:55:35.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
204692025-11-12T20:55:35.512ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204702025-11-12T20:55:35.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204712025-11-12T20:55:35.515ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204722025-11-12T20:55:35.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204732025-11-12T20:55:35.517ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
204742025-11-12T20:55:35.517ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
204752025-11-12T20:55:35.517ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 239.903689ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
204762025-11-12T20:55:35.519ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204772025-11-12T20:55:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204782025-11-12T20:55:35.524ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204792025-11-12T20:55:35.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204802025-11-12T20:55:35.530ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204812025-11-12T20:55:35.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204822025-11-12T20:55:35.534ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204832025-11-12T20:55:35.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204842025-11-12T20:55:35.540ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204852025-11-12T20:55:35.547ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
204862025-11-12T20:55:35.547ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204872025-11-12T20:55:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204882025-11-12T20:55:35.550ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204892025-11-12T20:55:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204902025-11-12T20:55:35.553ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
204912025-11-12T20:55:35.553ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
204922025-11-12T20:55:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
204932025-11-12T20:55:35.553ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 275.220202ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
204942025-11-12T20:55:35.554ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
204952025-11-12T20:55:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
204962025-11-12T20:55:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
204972025-11-12T20:55:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
204982025-11-12T20:55:35.556ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204992025-11-12T20:55:35.557ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205002025-11-12T20:55:35.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205012025-11-12T20:55:35.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
205022025-11-12T20:55:35.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205032025-11-12T20:55:35.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
205042025-11-12T20:55:35.562ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
205052025-11-12T20:55:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205062025-11-12T20:55:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
205072025-11-12T20:55:35.565ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
205082025-11-12T20:55:35.565ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205092025-11-12T20:55:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205102025-11-12T20:55:35.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205112025-11-12T20:55:35.572ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205122025-11-12T20:55:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205132025-11-12T20:55:35.575ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205142025-11-12T20:55:35.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205152025-11-12T20:55:35.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
205162025-11-12T20:55:35.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205172025-11-12T20:55:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205182025-11-12T20:55:35.578ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205192025-11-12T20:55:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205202025-11-12T20:55:35.582ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205212025-11-12T20:55:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205222025-11-12T20:55:35.585ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(e3133932-a041-4d2e-804a-0de86a112082, 0), (f7068698-2398-4367-9b5b-d2bbad127d66, 0), (278c2b12-b876-45e5-9268-e4580c375342, 0), (da575366-ca7a-460e-9ae6-b9457f2d2443, 0)] collection_id = e3133932-a041-4d2e-804a-0de86a112082
205232025-11-12T20:55:35.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
205242025-11-12T20:55:35.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
205252025-11-12T20:55:35.586ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205262025-11-12T20:55:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205272025-11-12T20:55:35.586ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
205282025-11-12T20:55:35.588ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205292025-11-12T20:55:35.588ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
205302025-11-12T20:55:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205312025-11-12T20:55:35.589ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 314.024997ms iteration = 21 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
205322025-11-12T20:55:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
205332025-11-12T20:55:35.589ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
205342025-11-12T20:55:35.590ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "00cb95be-6a2e-4353-b9cf-10a4bbeb1ead", "content-length": "78", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
205352025-11-12T20:55:35.590ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
205362025-11-12T20:55:35.590ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
205372025-11-12T20:55:35.590ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
205382025-11-12T20:55:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
205392025-11-12T20:55:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
205402025-11-12T20:55:35.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
205412025-11-12T20:55:35.591ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205422025-11-12T20:55:35.591ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
205432025-11-12T20:55:35.591ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
205442025-11-12T20:55:35.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205452025-11-12T20:55:35.592ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb08766f-9371-4e50-8aaa-9e0fae920dc6", "content-length": "26", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
205462025-11-12T20:55:35.592ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
205472025-11-12T20:55:35.593ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
205482025-11-12T20:55:35.593ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
205492025-11-12T20:55:35.593ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
205502025-11-12T20:55:35.594ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205512025-11-12T20:55:35.594ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
205522025-11-12T20:55:35.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
205532025-11-12T20:55:35.598ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
205542025-11-12T20:55:35.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
205552025-11-12T20:55:35.599ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
205562025-11-12T20:55:35.599ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
205572025-11-12T20:55:35.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
205582025-11-12T20:55:35.600ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
205592025-11-12T20:55:35.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
205602025-11-12T20:55:35.602ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
205612025-11-12T20:55:35.602ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
205622025-11-12T20:55:35.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
205632025-11-12T20:55:35.602ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
205642025-11-12T20:55:35.602ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
205652025-11-12T20:55:35.603ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "50121e35-e9ed-4460-a478-663dbbf73c36", "content-length": "81", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
205662025-11-12T20:55:35.604ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
205672025-11-12T20:55:35.604ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 329.23749ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
205682025-11-12T20:55:35.604ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
205692025-11-12T20:55:35.604ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205702025-11-12T20:55:35.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205712025-11-12T20:55:35.607ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205722025-11-12T20:55:35.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205732025-11-12T20:55:35.624ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205742025-11-12T20:55:35.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205752025-11-12T20:55:35.636ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205762025-11-12T20:55:35.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
205772025-11-12T20:55:35.645ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205782025-11-12T20:55:35.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205792025-11-12T20:55:35.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205802025-11-12T20:55:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205812025-11-12T20:55:35.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205822025-11-12T20:55:35.656ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
205832025-11-12T20:55:35.656ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205842025-11-12T20:55:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205852025-11-12T20:55:35.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205862025-11-12T20:55:35.658ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
205872025-11-12T20:55:35.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205882025-11-12T20:55:35.664ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205892025-11-12T20:55:35.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
205902025-11-12T20:55:35.669ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
205912025-11-12T20:55:35.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
205922025-11-12T20:55:35.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205932025-11-12T20:55:35.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205942025-11-12T20:55:35.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
205952025-11-12T20:55:35.673ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
205962025-11-12T20:55:35.673ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
205972025-11-12T20:55:35.673ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
205982025-11-12T20:55:35.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205992025-11-12T20:55:35.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206002025-11-12T20:55:35.675ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
206012025-11-12T20:55:35.675ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
206022025-11-12T20:55:35.676ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d1ea758-ae87-4592-bf28-f913ee08d4ee", "content-length": "81", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206032025-11-12T20:55:35.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206042025-11-12T20:55:35.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
206052025-11-12T20:55:35.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206062025-11-12T20:55:35.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206072025-11-12T20:55:35.685ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
206082025-11-12T20:55:35.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206092025-11-12T20:55:35.685ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
206102025-11-12T20:55:35.686ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "4309e760-8e50-4dd0-b2a1-04c7948a726e", "content-length": "81", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206112025-11-12T20:55:35.691ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206122025-11-12T20:55:35.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206132025-11-12T20:55:35.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206142025-11-12T20:55:35.692ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 414.25327ms iteration = 21 status = {"disabled":false,"errors":[],"sps":[]}
206152025-11-12T20:55:35.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206162025-11-12T20:55:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206172025-11-12T20:55:35.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206182025-11-12T20:55:35.704ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
206192025-11-12T20:55:35.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206202025-11-12T20:55:35.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206212025-11-12T20:55:35.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206222025-11-12T20:55:35.710ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
206232025-11-12T20:55:35.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206242025-11-12T20:55:35.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206252025-11-12T20:55:35.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
206262025-11-12T20:55:35.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206272025-11-12T20:55:35.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206282025-11-12T20:55:35.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206292025-11-12T20:55:35.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206302025-11-12T20:55:35.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206312025-11-12T20:55:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
206322025-11-12T20:55:35.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206332025-11-12T20:55:35.728ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
206342025-11-12T20:55:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206352025-11-12T20:55:35.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
206362025-11-12T20:55:35.732ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206372025-11-12T20:55:35.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
206382025-11-12T20:55:35.739ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
206392025-11-12T20:55:35.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
206402025-11-12T20:55:35.741ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
206412025-11-12T20:55:35.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
206422025-11-12T20:55:35.745ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206432025-11-12T20:55:35.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
206442025-11-12T20:55:35.749ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
206452025-11-12T20:55:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
206462025-11-12T20:55:35.755ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
206472025-11-12T20:55:35.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
206482025-11-12T20:55:35.757ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206492025-11-12T20:55:35.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
206502025-11-12T20:55:35.761ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
206512025-11-12T20:55:35.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
206522025-11-12T20:55:35.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206532025-11-12T20:55:35.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206542025-11-12T20:55:35.767ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
206552025-11-12T20:55:35.767ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
206562025-11-12T20:55:35.767ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "525c1031-732e-4885-a1c6-d194489ce79f", "content-length": "372", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206572025-11-12T20:55:35.767ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
206582025-11-12T20:55:35.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
206592025-11-12T20:55:35.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206602025-11-12T20:55:35.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206612025-11-12T20:55:35.772ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
206622025-11-12T20:55:35.772ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "2b7724aa-f998-41fc-a4bb-97a418a67eb4", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206632025-11-12T20:55:35.772ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
206642025-11-12T20:55:35.773ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "a29a0b1f-94b5-493c-84d6-2efc9dd1a230", "content-length": "26", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206652025-11-12T20:55:35.773ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
206662025-11-12T20:55:35.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206672025-11-12T20:55:35.775ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 497.2908ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"cdf575e5-7c27-4aa3-b1ec-77468da13285","time":"2025-11-12T20:50:35.842248705Z"},{"operation":"list","target_sled":"cdf575e5-7c27-4aa3-b1ec-77468da13285","time":"2025-11-12T20:50:35.842479851Z"},{"generation":1,"operation":"put_config","target_sled":"cdf575e5-7c27-4aa3-b1ec-77468da13285","time":"2025-11-12T20:55:35.772905565Z"},{"operation":"list","target_sled":"cdf575e5-7c27-4aa3-b1ec-77468da13285","time":"2025-11-12T20:55:35.773201332Z"}]}
206682025-11-12T20:55:35.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206692025-11-12T20:55:35.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206702025-11-12T20:55:35.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206712025-11-12T20:55:35.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206722025-11-12T20:55:35.785ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
206732025-11-12T20:55:35.785ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
206742025-11-12T20:55:35.787ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206752025-11-12T20:55:35.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206762025-11-12T20:55:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206772025-11-12T20:55:35.790ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
206782025-11-12T20:55:35.790ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
206792025-11-12T20:55:35.790ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5ef7d442-0059-4a63-80fa-eaed8fb626d5", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206802025-11-12T20:55:35.790ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
206812025-11-12T20:55:35.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206822025-11-12T20:55:35.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
206832025-11-12T20:55:35.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206842025-11-12T20:55:35.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206852025-11-12T20:55:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206862025-11-12T20:55:35.794ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
206872025-11-12T20:55:35.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
206882025-11-12T20:55:35.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206892025-11-12T20:55:35.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
206902025-11-12T20:55:35.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206912025-11-12T20:55:35.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206922025-11-12T20:55:35.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
206932025-11-12T20:55:35.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206942025-11-12T20:55:35.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206952025-11-12T20:55:35.808ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
206962025-11-12T20:55:35.808ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
206972025-11-12T20:55:35.809ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ec88828b-f7f9-4cee-92c2-54890f4b5e72", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
206982025-11-12T20:55:35.812ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 534.171134ms iteration = 31 status = {}
206992025-11-12T20:55:35.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
207002025-11-12T20:55:35.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207012025-11-12T20:55:35.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207022025-11-12T20:55:35.816ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
207032025-11-12T20:55:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
207042025-11-12T20:55:35.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207052025-11-12T20:55:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207062025-11-12T20:55:35.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207072025-11-12T20:55:35.818ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207082025-11-12T20:55:35.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207092025-11-12T20:55:35.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207102025-11-12T20:55:35.822ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
207112025-11-12T20:55:35.822ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
207122025-11-12T20:55:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
207132025-11-12T20:55:35.825ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207142025-11-12T20:55:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
207152025-11-12T20:55:35.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207162025-11-12T20:55:35.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207172025-11-12T20:55:35.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207182025-11-12T20:55:35.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207192025-11-12T20:55:35.829ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
207202025-11-12T20:55:35.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
207212025-11-12T20:55:35.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
207222025-11-12T20:55:35.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207232025-11-12T20:55:35.834ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
207242025-11-12T20:55:35.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207252025-11-12T20:55:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207262025-11-12T20:55:35.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207272025-11-12T20:55:35.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207282025-11-12T20:55:35.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207292025-11-12T20:55:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
207302025-11-12T20:55:35.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207312025-11-12T20:55:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207322025-11-12T20:55:35.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207332025-11-12T20:55:35.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207342025-11-12T20:55:35.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207352025-11-12T20:55:35.850ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
207362025-11-12T20:55:35.851ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "1bfad670-15a6-4f5c-af38-d43dc2724e7b", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
207372025-11-12T20:55:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
207382025-11-12T20:55:35.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207392025-11-12T20:55:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207402025-11-12T20:55:35.859ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = e3133932-a041-4d2e-804a-0de86a112082 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
207412025-11-12T20:55:35.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
207422025-11-12T20:55:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
207432025-11-12T20:55:35.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207442025-11-12T20:55:35.859ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = FirewallRules step id = Ensure
207452025-11-12T20:55:35.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207462025-11-12T20:55:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207472025-11-12T20:55:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207482025-11-12T20:55:35.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207492025-11-12T20:55:35.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207502025-11-12T20:55:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207512025-11-12T20:55:35.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207522025-11-12T20:55:35.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207532025-11-12T20:55:35.864ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(f7068698-2398-4367-9b5b-d2bbad127d66, 0), (278c2b12-b876-45e5-9268-e4580c375342, 0), (da575366-ca7a-460e-9ae6-b9457f2d2443, 0)]
207542025-11-12T20:55:35.865ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 362206365 } })
207552025-11-12T20:55:35.866ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
207562025-11-12T20:55:35.866ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 587.896523ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
207572025-11-12T20:55:35.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207582025-11-12T20:55:35.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207592025-11-12T20:55:35.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
207602025-11-12T20:55:35.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207612025-11-12T20:55:35.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207622025-11-12T20:55:35.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207632025-11-12T20:55:35.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207642025-11-12T20:55:35.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207652025-11-12T20:55:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207662025-11-12T20:55:35.869ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
207672025-11-12T20:55:35.869ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
207682025-11-12T20:55:35.870ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd53a9d7-81f8-450d-ae40-98d1df5d8c57", "content-length": "26", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
207692025-11-12T20:55:35.870ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
207702025-11-12T20:55:35.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207712025-11-12T20:55:35.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207722025-11-12T20:55:35.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207732025-11-12T20:55:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207742025-11-12T20:55:35.872ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
207752025-11-12T20:55:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
207762025-11-12T20:55:35.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207772025-11-12T20:55:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207782025-11-12T20:55:35.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
207792025-11-12T20:55:35.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207802025-11-12T20:55:35.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207812025-11-12T20:55:35.882ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
207822025-11-12T20:55:35.882ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
207832025-11-12T20:55:35.882ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 3b5f4dc9-465c-4b56-bd15-f32cd63d1390 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
207842025-11-12T20:55:35.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207852025-11-12T20:55:35.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207862025-11-12T20:55:35.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207872025-11-12T20:55:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207882025-11-12T20:55:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207892025-11-12T20:55:35.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207902025-11-12T20:55:35.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207912025-11-12T20:55:35.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207922025-11-12T20:55:35.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
207932025-11-12T20:55:35.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207942025-11-12T20:55:35.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207952025-11-12T20:55:35.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
207962025-11-12T20:55:35.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
207972025-11-12T20:55:35.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
207982025-11-12T20:55:35.895ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
207992025-11-12T20:55:35.895ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
208002025-11-12T20:55:35.895ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 617.300844ms iteration = 25 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
208012025-11-12T20:55:35.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208022025-11-12T20:55:35.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208032025-11-12T20:55:35.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208042025-11-12T20:55:35.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208052025-11-12T20:55:35.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208062025-11-12T20:55:35.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208072025-11-12T20:55:35.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208082025-11-12T20:55:35.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208092025-11-12T20:55:35.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208102025-11-12T20:55:35.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208112025-11-12T20:55:35.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208122025-11-12T20:55:35.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208132025-11-12T20:55:35.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208142025-11-12T20:55:35.917ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
208152025-11-12T20:55:35.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208162025-11-12T20:55:35.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208172025-11-12T20:55:35.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208182025-11-12T20:55:35.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208192025-11-12T20:55:35.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208202025-11-12T20:55:35.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208212025-11-12T20:55:35.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208222025-11-12T20:55:35.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208232025-11-12T20:55:35.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208242025-11-12T20:55:35.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208252025-11-12T20:55:35.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208262025-11-12T20:55:35.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208272025-11-12T20:55:35.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208282025-11-12T20:55:35.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208292025-11-12T20:55:35.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208302025-11-12T20:55:35.940ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
208312025-11-12T20:55:35.940ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c8fa16c9-5ab9-4576-a3ed-2306b5d36fa9", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
208322025-11-12T20:55:35.940ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
208332025-11-12T20:55:35.943ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "b0100d65-3d03-4024-af34-3b8089fd5b11", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
208342025-11-12T20:55:35.943ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
208352025-11-12T20:55:35.944ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
208362025-11-12T20:55:35.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208372025-11-12T20:55:35.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208382025-11-12T20:55:35.945ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
208392025-11-12T20:55:35.945ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 670.377978ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
208402025-11-12T20:55:35.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208412025-11-12T20:55:35.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208422025-11-12T20:55:35.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208432025-11-12T20:55:35.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208442025-11-12T20:55:35.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208452025-11-12T20:55:35.955ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
208462025-11-12T20:55:35.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208472025-11-12T20:55:35.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208482025-11-12T20:55:35.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208492025-11-12T20:55:35.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208502025-11-12T20:55:35.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208512025-11-12T20:55:35.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208522025-11-12T20:55:35.962ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
208532025-11-12T20:55:35.963ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "525e0e12-3172-48dd-81d2-502e82512c66", "date": "Wed, 12 Nov 2025 20:55:35 GMT"} })
208542025-11-12T20:55:35.963ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
208552025-11-12T20:55:35.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208562025-11-12T20:55:35.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208572025-11-12T20:55:35.965ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
208582025-11-12T20:55:35.965ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 675.016198ms iteration = 11 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
208592025-11-12T20:55:35.965ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Dependency
208602025-11-12T20:55:35.965ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
208612025-11-12T20:55:35.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
208622025-11-12T20:55:35.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
208632025-11-12T20:55:35.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208642025-11-12T20:55:35.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208652025-11-12T20:55:35.967ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208662025-11-12T20:55:35.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208672025-11-12T20:55:35.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208682025-11-12T20:55:35.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208692025-11-12T20:55:35.970ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
208702025-11-12T20:55:35.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
208712025-11-12T20:55:35.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
208722025-11-12T20:55:35.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208732025-11-12T20:55:35.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208742025-11-12T20:55:35.972ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208752025-11-12T20:55:35.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208762025-11-12T20:55:35.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208772025-11-12T20:55:35.976ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
208782025-11-12T20:55:35.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
208792025-11-12T20:55:35.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
208802025-11-12T20:55:35.979ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
208812025-11-12T20:55:35.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208822025-11-12T20:55:35.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208832025-11-12T20:55:35.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208842025-11-12T20:55:35.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208852025-11-12T20:55:35.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208862025-11-12T20:55:35.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208872025-11-12T20:55:35.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208882025-11-12T20:55:35.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
208892025-11-12T20:55:35.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208902025-11-12T20:55:35.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208912025-11-12T20:55:35.997ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
208922025-11-12T20:55:35.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208932025-11-12T20:55:35.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208942025-11-12T20:55:35.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208952025-11-12T20:55:36.001ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
208962025-11-12T20:55:36.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
208972025-11-12T20:55:36.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
208982025-11-12T20:55:36.003ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
208992025-11-12T20:55:36.003ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
209002025-11-12T20:55:36.003ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
209012025-11-12T20:55:36.081ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
209022025-11-12T20:55:36.081ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
209032025-11-12T20:55:36.082ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 806.752637ms iteration = 21 status = {}
209042025-11-12T20:55:36.126ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 623287772 } })
209052025-11-12T20:55:36.126ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
209062025-11-12T20:55:36.144ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 640780085 } })
209072025-11-12T20:55:36.144ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
209082025-11-12T20:55:36.144ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
209092025-11-12T20:55:36.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
209102025-11-12T20:55:36.145ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "54978928-2e43-4017-a243-8d8cc2774469", "content-length": "476", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209112025-11-12T20:55:36.145ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
209122025-11-12T20:55:36.145ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aa75b48-2ec2-4802-a694-42bfbf545166", "content-length": "734", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209132025-11-12T20:55:36.145ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
209142025-11-12T20:55:36.146ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fab67c1-893e-4e6b-a670-37e1419b7edf", "content-length": "111", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209152025-11-12T20:55:36.146ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
209162025-11-12T20:55:36.146ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8672310-ac3f-400f-b028-ddab83d47b35", "content-length": "111", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209172025-11-12T20:55:36.146ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
209182025-11-12T20:55:36.146ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51d4779e-8299-4324-8ba9-b2102fef8c70", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209192025-11-12T20:55:36.146ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
209202025-11-12T20:55:36.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34ec5fe2-46a5-4ac0-aa27-6c927477d56a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209212025-11-12T20:55:36.147ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
209222025-11-12T20:55:36.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9f7c6df-1684-4c44-b580-18b6f3537e1d", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209232025-11-12T20:55:36.147ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
209242025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e7dc3f1-3b11-4790-b3bd-854a5b3c68cb", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209252025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
209262025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "faddce1f-7b35-4cdb-9157-a51ef07ff572", "content-length": "702", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209272025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
209282025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d85fe31b-4ded-4fa9-8a43-cfa5f980e757", "content-length": "727", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209292025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
209302025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "458068b1-40a1-457b-88ea-d9f236bc6a94", "content-length": "729", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209312025-11-12T20:55:36.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
209322025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dee6c60-4f58-458b-aa6f-a061878f5da9", "content-length": "728", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209332025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
209342025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a2c3f82-7455-4e59-9b77-816ea1241633", "content-length": "722", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209352025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
209362025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b7d09c1-b593-4a8c-8531-478f309e9888", "content-length": "10", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209372025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
209382025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c900f52d-41e3-4ca2-9892-9ef40e0f8cbb", "content-length": "144", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209392025-11-12T20:55:36.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
209402025-11-12T20:55:36.150ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5990bed0-34b4-4e42-a154-10cd0aeee4f6", "content-length": "147", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209412025-11-12T20:55:36.150ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
209422025-11-12T20:55:36.150ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1658e026-d2be-4994-bee6-c4585993521a", "content-length": "109", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209432025-11-12T20:55:36.150ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
209442025-11-12T20:55:36.151ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "089eee56-056b-4c62-8fc2-492aba67fb47", "content-length": "109", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209452025-11-12T20:55:36.151ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
209462025-11-12T20:55:36.151ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1d23568-89b6-4b73-96e0-7da9b8def226", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209472025-11-12T20:55:36.151ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
209482025-11-12T20:55:36.152ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f866bc8b-e93e-47d5-a128-c25f9b5a354a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209492025-11-12T20:55:36.152ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
209502025-11-12T20:55:36.152ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c37746de-d2b6-462b-8c56-6ec026144130", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209512025-11-12T20:55:36.152ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
209522025-11-12T20:55:36.152ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4aacc3ba-d96b-4d65-a287-b6f5ecd3e77b", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209532025-11-12T20:55:36.152ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
209542025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f07f465a-3b85-4670-82cd-6e97dd14fb28", "content-length": "702", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209552025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
209562025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42e3f8b9-cb03-4591-9a7f-6a4671462d99", "content-length": "727", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209572025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
209582025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cfae6ad-5398-4bfe-8e51-f82a0e85f3b7", "content-length": "729", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209592025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
209602025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe3d4770-2bb8-466e-87be-2be710af0254", "content-length": "728", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209612025-11-12T20:55:36.153ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
209622025-11-12T20:55:36.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
209632025-11-12T20:55:36.156ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209642025-11-12T20:55:36.158ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
209652025-11-12T20:55:36.158ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
209662025-11-12T20:55:36.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
209672025-11-12T20:55:36.160ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209682025-11-12T20:55:36.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
209692025-11-12T20:55:36.164ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
209702025-11-12T20:55:36.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
209712025-11-12T20:55:36.184ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
209722025-11-12T20:55:36.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
209732025-11-12T20:55:36.186ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
209742025-11-12T20:55:36.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
209752025-11-12T20:55:36.187ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209762025-11-12T20:55:36.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
209772025-11-12T20:55:36.190ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
209782025-11-12T20:55:36.191ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
209792025-11-12T20:55:36.192ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "e53e5b50-f789-4d9c-9dc8-e62e7e5add05", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209802025-11-12T20:55:36.192ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b469553-5746-4a47-9b24-2ee9915952b2", "content-length": "691", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209812025-11-12T20:55:36.192ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
209822025-11-12T20:55:36.192ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
209832025-11-12T20:55:36.192ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
209842025-11-12T20:55:36.194ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
209852025-11-12T20:55:36.194ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
209862025-11-12T20:55:36.194ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
209872025-11-12T20:55:36.194ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 916.642751ms
209882025-11-12T20:55:36.194ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 916.653661ms iteration = 3 status = {}
209892025-11-12T20:55:36.195ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 916.977189ms iteration = 21 status = {}
209902025-11-12T20:55:36.201ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "574d3df0-6cc9-42d1-b498-2045de9f686c", "content-length": "372", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209912025-11-12T20:55:36.201ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
209922025-11-12T20:55:36.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
209932025-11-12T20:55:36.214ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
209942025-11-12T20:55:36.214ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
209952025-11-12T20:55:36.215ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e4c98d7a-5049-42c2-8101-4fa8f148df29", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
209962025-11-12T20:55:36.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
209972025-11-12T20:55:36.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
209982025-11-12T20:55:36.217ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
209992025-11-12T20:55:36.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
210002025-11-12T20:55:36.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
210012025-11-12T20:55:36.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
210022025-11-12T20:55:36.223ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
210032025-11-12T20:55:36.223ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
210042025-11-12T20:55:36.224ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b4b60023-bc04-4d0a-bd83-61ee0beac538", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210052025-11-12T20:55:36.226ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 260.872801ms iteration = 32 status = {}
210062025-11-12T20:55:36.308ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
210072025-11-12T20:55:36.308ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
210082025-11-12T20:55:36.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210092025-11-12T20:55:36.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210102025-11-12T20:55:36.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210112025-11-12T20:55:36.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210122025-11-12T20:55:36.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210132025-11-12T20:55:36.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
210142025-11-12T20:55:36.358ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
210152025-11-12T20:55:36.359ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "aa87d7a6-9444-4d67-82db-7bfe62d8165f", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210162025-11-12T20:55:36.359ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
210172025-11-12T20:55:36.360ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d167fc87-2654-4c9d-9e45-8ad86af323e7", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210182025-11-12T20:55:36.360ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210192025-11-12T20:55:36.362ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
210202025-11-12T20:55:36.362ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = Dns step id = Ensure
210212025-11-12T20:55:36.362ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
210222025-11-12T20:55:36.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210232025-11-12T20:55:36.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210242025-11-12T20:55:36.362ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5ba7229d-054c-47b3-aa79-e42fd6a0e005", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210252025-11-12T20:55:36.362ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210262025-11-12T20:55:36.362ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7d5e1e9-69dd-4ce9-95bf-834c732b6675", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210272025-11-12T20:55:36.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210282025-11-12T20:55:36.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210292025-11-12T20:55:36.364ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ab31b20a-1924-467f-8b8a-f5fdac18cb7f", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210302025-11-12T20:55:36.364ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210312025-11-12T20:55:36.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210322025-11-12T20:55:36.365ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
210332025-11-12T20:55:36.365ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
210342025-11-12T20:55:36.365ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
210352025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
210362025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
210372025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
210382025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
210392025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
210402025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f87fba16-196c-4de9-aca3-3e633c76f9f5", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210412025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
210422025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec2d3c6d-a88b-4bb9-8992-76a276f94a05", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210432025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "edeaffe8-891b-431b-846a-6aa2bdb9e8de", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210442025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a7144e1-ffc4-4636-bd3e-0ee072b41e8f", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210452025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
210462025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
210472025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
210482025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f1ee377-898d-44a6-8108-3465609f4aef", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
210492025-11-12T20:55:36.365ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
210502025-11-12T20:55:36.366ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
210512025-11-12T20:55:36.366ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 1.090979653s iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
210522025-11-12T20:55:36.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
210532025-11-12T20:55:36.369ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
210542025-11-12T20:55:36.369ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
210552025-11-12T20:55:36.369ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
210562025-11-12T20:55:36.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210572025-11-12T20:55:36.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210582025-11-12T20:55:36.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210592025-11-12T20:55:36.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210602025-11-12T20:55:36.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210612025-11-12T20:55:36.373ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
210622025-11-12T20:55:36.373ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210632025-11-12T20:55:36.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210642025-11-12T20:55:36.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
210652025-11-12T20:55:36.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210662025-11-12T20:55:36.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210672025-11-12T20:55:36.377ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
210682025-11-12T20:55:36.378ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
210692025-11-12T20:55:36.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210702025-11-12T20:55:36.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210712025-11-12T20:55:36.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210722025-11-12T20:55:36.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
210732025-11-12T20:55:36.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210742025-11-12T20:55:36.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210752025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
210762025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
210772025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
210782025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
210792025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
210802025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
210812025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
210822025-11-12T20:55:36.387ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
210832025-11-12T20:55:36.388ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
210842025-11-12T20:55:36.388ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
210852025-11-12T20:55:36.388ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
210862025-11-12T20:55:36.388ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
210872025-11-12T20:55:36.388ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
210882025-11-12T20:55:36.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210892025-11-12T20:55:36.388ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
210902025-11-12T20:55:36.388ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210912025-11-12T20:55:36.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210922025-11-12T20:55:36.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210932025-11-12T20:55:36.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
210942025-11-12T20:55:36.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
210952025-11-12T20:55:36.391ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
210962025-11-12T20:55:36.391ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
210972025-11-12T20:55:36.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210982025-11-12T20:55:36.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
210992025-11-12T20:55:36.397ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
211002025-11-12T20:55:36.397ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
211012025-11-12T20:55:36.397ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
211022025-11-12T20:55:36.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211032025-11-12T20:55:36.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211042025-11-12T20:55:36.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211052025-11-12T20:55:36.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
211062025-11-12T20:55:36.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211072025-11-12T20:55:36.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211082025-11-12T20:55:36.406ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
211092025-11-12T20:55:36.406ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
211102025-11-12T20:55:36.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211112025-11-12T20:55:36.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211122025-11-12T20:55:36.410ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "40f0d583-b090-48f1-b0dc-46440b011612", "content-length": "415987", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211132025-11-12T20:55:36.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211142025-11-12T20:55:36.414ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3b0bbab-01f7-4650-a8c4-d6657bd2b8e0", "content-length": "407674", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211152025-11-12T20:55:36.414ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b1aec52-ef3f-43ff-994a-613d5e872e41", "content-length": "411436", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211162025-11-12T20:55:36.415ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c67dab24-73a7-4e83-b6e2-71ba24d40557", "content-length": "435733", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211172025-11-12T20:55:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
211182025-11-12T20:55:36.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211192025-11-12T20:55:36.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211202025-11-12T20:55:36.418ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
211212025-11-12T20:55:36.418ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
211222025-11-12T20:55:36.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211232025-11-12T20:55:36.419ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c05b3643-1d49-400c-93aa-a89a6fd4a9ce", "content-length": "406508", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211242025-11-12T20:55:36.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211252025-11-12T20:55:36.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211262025-11-12T20:55:36.421ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
211272025-11-12T20:55:36.421ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
211282025-11-12T20:55:36.421ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
211292025-11-12T20:55:36.421ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
211302025-11-12T20:55:36.421ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
211312025-11-12T20:55:36.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
211322025-11-12T20:55:36.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211332025-11-12T20:55:36.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211342025-11-12T20:55:36.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211352025-11-12T20:55:36.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211362025-11-12T20:55:36.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211372025-11-12T20:55:36.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211382025-11-12T20:55:36.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211392025-11-12T20:55:36.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
211402025-11-12T20:55:36.433ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
211412025-11-12T20:55:36.433ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
211422025-11-12T20:55:36.433ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = OmicronZones step id = Cleanup
211432025-11-12T20:55:36.433ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = OmicronZones step id = Cleanup
211442025-11-12T20:55:36.433ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = PhysicalDisks step id = Cleanup
211452025-11-12T20:55:36.433ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = Clickhouse step id = Ensure
211462025-11-12T20:55:36.433ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = Clickhouse step id = Ensure
211472025-11-12T20:55:36.460ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
211482025-11-12T20:55:36.460ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
211492025-11-12T20:55:36.464ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "075519c6-a503-46be-9611-6d105f1a1228", "content-length": "127", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211502025-11-12T20:55:36.466ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
211512025-11-12T20:55:36.471ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
211522025-11-12T20:55:36.476ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "14b72db5-500b-4a18-9426-2ca26e0988cf", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211532025-11-12T20:55:36.479ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = SupportBundles step id = Cleanup
211542025-11-12T20:55:36.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211552025-11-12T20:55:36.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211562025-11-12T20:55:36.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211572025-11-12T20:55:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
211582025-11-12T20:55:36.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211592025-11-12T20:55:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211602025-11-12T20:55:36.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211612025-11-12T20:55:36.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211622025-11-12T20:55:36.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211632025-11-12T20:55:36.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
211642025-11-12T20:55:36.496ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = OmicronZones step id = Cleanup
211652025-11-12T20:55:36.496ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
211662025-11-12T20:55:36.496ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
211672025-11-12T20:55:36.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211682025-11-12T20:55:36.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211692025-11-12T20:55:36.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211702025-11-12T20:55:36.500ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
211712025-11-12T20:55:36.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211722025-11-12T20:55:36.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211732025-11-12T20:55:36.502ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
211742025-11-12T20:55:36.502ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
211752025-11-12T20:55:36.502ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = Cockroach step id = Ensure
211762025-11-12T20:55:36.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
211772025-11-12T20:55:36.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
211782025-11-12T20:55:36.520ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
211792025-11-12T20:55:36.520ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c982fdce-0093-4829-a6bd-f8f8c7bd1de3 step component = MgsUpdates step id = Ensure
211802025-11-12T20:55:36.520ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 32 reason = Dependency
211812025-11-12T20:55:36.520ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
211822025-11-12T20:55:36.520ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.245110971s iteration = 12 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3260,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":149534550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":149534500,"secs":0}},"event_index":1,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":149550740,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":23467997,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":23467937,"secs":0}},"event_index":2,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":173033948,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":25270574,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":25270534,"secs":0}},"event_index":3,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":198323352,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":385746141,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":385746092,"secs":0}},"event_index":4,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":584088555,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":502524735,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":502524685,"secs":0}},"event_index":5,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":86637550,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":71716276,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":71716236,"secs":0}},"event_index":6,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158367816,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4410,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4370,"secs":0}},"event_index":7,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158382227,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1960,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":1920,"secs":0}},"event_index":8,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158386327,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3220,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3190,"secs":0}},"event_index":9,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158391407,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1910,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1870,"secs":0}},"event_index":10,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158394937,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":45287772,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":45287732,"secs":0}},"event_index":11,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":203684089,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":17586806,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":17586495,"secs":0}},"event_index":12,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":221283774,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5935453,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5935413,"secs":0}},"event_index":13,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":227231078,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":17427201,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":17427161,"secs":0}},"event_index":14,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":244669840,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5230,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5180,"secs":0}},"event_index":15,"execution_id":"dd2093a4-a97d-460c-bbdb-c52d9709d9e7","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":244686270,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
211832025-11-12T20:55:36.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
211842025-11-12T20:55:36.520ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "15a346d7-1eb8-4422-9ce4-897f710a27a3", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211852025-11-12T20:55:36.520ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
211862025-11-12T20:55:36.520ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
211872025-11-12T20:55:36.522ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211882025-11-12T20:55:36.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
211892025-11-12T20:55:36.524ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
211902025-11-12T20:55:36.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
211912025-11-12T20:55:36.526ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc129c02-bdb4-43d8-987e-c1811a1f6cbc", "content-length": "682", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
211922025-11-12T20:55:36.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
211932025-11-12T20:55:36.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
211942025-11-12T20:55:36.527ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
211952025-11-12T20:55:36.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
211962025-11-12T20:55:36.529ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211972025-11-12T20:55:36.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
211982025-11-12T20:55:36.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
211992025-11-12T20:55:36.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
212002025-11-12T20:55:36.531ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
212012025-11-12T20:55:36.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
212022025-11-12T20:55:36.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
212032025-11-12T20:55:36.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
212042025-11-12T20:55:36.533ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
212052025-11-12T20:55:36.533ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 13.329475ms iteration = 32 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
212062025-11-12T20:55:36.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
212072025-11-12T20:55:36.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
212082025-11-12T20:55:36.541ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.264022281s iteration = 21 status = {}
212092025-11-12T20:55:36.568ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
212102025-11-12T20:55:36.568ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
212112025-11-12T20:55:36.569ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f8738c-4e8f-4f68-bd34-58ec18385acf", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
212122025-11-12T20:55:36.569ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
212132025-11-12T20:55:36.570ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
212142025-11-12T20:55:36.570ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
212152025-11-12T20:55:36.571ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcfd28e5-be6b-4d54-a8d1-e0a6b3a6924d", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
212162025-11-12T20:55:36.571ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
212172025-11-12T20:55:36.573ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
212182025-11-12T20:55:36.573ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
212192025-11-12T20:55:36.573ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f808be6e-2756-442c-98e3-ccc691a39d58", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:55:36 GMT"} })
212202025-11-12T20:55:36.574ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
212212025-11-12T20:55:36.576ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
212222025-11-12T20:55:36.576ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
212232025-11-12T20:55:36.578ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
212242025-11-12T20:55:36.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
212252025-11-12T20:55:36.580ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212262025-11-12T20:55:36.582ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
212272025-11-12T20:55:36.583ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
212282025-11-12T20:55:36.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
212292025-11-12T20:55:36.798ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
212302025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0 time_started = 2025-11-12 20:55:36.144642 UTC
212312025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 53 reason = Dependency
212322025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
212332025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.521880754s iteration = 12 status = {"collection_id":"8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0","time_done":"2025-11-12 20:55:36.576287 UTC","time_started":"2025-11-12 20:55:36.144642 UTC"}
212342025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Dependency
212352025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
212362025-11-12T20:55:36.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
212372025-11-12T20:55:36.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
212382025-11-12T20:55:36.800ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212392025-11-12T20:55:36.800ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212402025-11-12T20:55:36.803ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
212412025-11-12T20:55:36.803ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
212422025-11-12T20:55:36.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
212432025-11-12T20:55:36.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
212442025-11-12T20:55:36.804ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
212452025-11-12T20:55:36.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
212462025-11-12T20:55:36.895ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
212472025-11-12T20:55:36.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
212482025-11-12T20:55:36.895ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(278c2b12-b876-45e5-9268-e4580c375342, 0), (da575366-ca7a-460e-9ae6-b9457f2d2443, 0), (d09e62cf-5276-4e20-ace2-c56774b2adea, 0), (8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0, 0)] collection_id = 278c2b12-b876-45e5-9268-e4580c375342
212492025-11-12T20:55:36.895ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
212502025-11-12T20:55:36.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
212512025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Dependency
212522025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 157.248529ms iteration = 53 status = {"Loaded":{"collection_id":"fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51","time_loaded":"2025-11-12T20:55:36.798505725Z","time_started":"2025-11-12T20:55:36.236034Z"}}
212532025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 16 reason = Dependency
212542025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 26 reason = Dependency
212552025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
212562025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
212572025-11-12T20:55:36.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
212582025-11-12T20:55:36.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
212592025-11-12T20:55:36.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
212602025-11-12T20:55:36.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
212612025-11-12T20:55:36.957ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212622025-11-12T20:55:36.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212632025-11-12T20:55:36.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212642025-11-12T20:55:36.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
212652025-11-12T20:55:36.960ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
212662025-11-12T20:55:36.960ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
212672025-11-12T20:55:36.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
212682025-11-12T20:55:36.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
212692025-11-12T20:55:36.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
212702025-11-12T20:55:36.961ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
212712025-11-12T20:55:36.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
212722025-11-12T20:55:36.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
212732025-11-12T20:55:36.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212742025-11-12T20:55:36.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212752025-11-12T20:55:36.963ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
212762025-11-12T20:55:36.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
212772025-11-12T20:55:36.965ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212782025-11-12T20:55:36.966ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
212792025-11-12T20:55:36.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
212802025-11-12T20:55:36.966ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
212812025-11-12T20:55:36.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
212822025-11-12T20:55:36.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
212832025-11-12T20:55:36.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
212842025-11-12T20:55:36.968ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
212852025-11-12T20:55:36.968ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
212862025-11-12T20:55:36.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
212872025-11-12T20:55:36.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
212882025-11-12T20:55:36.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
212892025-11-12T20:55:36.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
212902025-11-12T20:55:36.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
212912025-11-12T20:55:36.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212922025-11-12T20:55:36.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212932025-11-12T20:55:36.972ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
212942025-11-12T20:55:36.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
212952025-11-12T20:55:36.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
212962025-11-12T20:55:36.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
212972025-11-12T20:55:36.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
212982025-11-12T20:55:36.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
212992025-11-12T20:55:36.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213002025-11-12T20:55:36.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213012025-11-12T20:55:36.975ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
213022025-11-12T20:55:36.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213032025-11-12T20:55:36.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213042025-11-12T20:55:36.977ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
213052025-11-12T20:55:36.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
213062025-11-12T20:55:36.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
213072025-11-12T20:55:36.979ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
213082025-11-12T20:55:36.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
213092025-11-12T20:55:36.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
213102025-11-12T20:55:37.011ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 55.835405ms iteration = 34 status = {"physical_disks_added":0}
213112025-11-12T20:55:37.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213122025-11-12T20:55:37.024ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
213132025-11-12T20:55:37.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
213142025-11-12T20:55:37.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
213152025-11-12T20:55:37.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213162025-11-12T20:55:37.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
213172025-11-12T20:55:37.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
213182025-11-12T20:55:37.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
213192025-11-12T20:55:37.029ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
213202025-11-12T20:55:37.029ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 73.491252ms iteration = 16 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
213212025-11-12T20:55:37.031ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 278c2b12-b876-45e5-9268-e4580c375342 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
213222025-11-12T20:55:37.031ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
213232025-11-12T20:55:37.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
213242025-11-12T20:55:37.036ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(da575366-ca7a-460e-9ae6-b9457f2d2443, 0), (d09e62cf-5276-4e20-ace2-c56774b2adea, 0), (8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0, 0), (fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51, 0)] collection_id = da575366-ca7a-460e-9ae6-b9457f2d2443
213252025-11-12T20:55:37.036ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
213262025-11-12T20:55:37.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
213272025-11-12T20:55:37.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213282025-11-12T20:55:37.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213292025-11-12T20:55:37.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213302025-11-12T20:55:37.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213312025-11-12T20:55:37.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213322025-11-12T20:55:37.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
213332025-11-12T20:55:37.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213342025-11-12T20:55:37.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213352025-11-12T20:55:37.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213362025-11-12T20:55:37.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213372025-11-12T20:55:37.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213382025-11-12T20:55:37.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
213392025-11-12T20:55:37.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213402025-11-12T20:55:37.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213412025-11-12T20:55:37.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213422025-11-12T20:55:37.076ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
213432025-11-12T20:55:37.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213442025-11-12T20:55:37.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213452025-11-12T20:55:37.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213462025-11-12T20:55:37.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213472025-11-12T20:55:37.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213482025-11-12T20:55:37.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
213492025-11-12T20:55:37.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213502025-11-12T20:55:37.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213512025-11-12T20:55:37.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213522025-11-12T20:55:37.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213532025-11-12T20:55:37.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213542025-11-12T20:55:37.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
213552025-11-12T20:55:37.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213562025-11-12T20:55:37.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213572025-11-12T20:55:37.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213582025-11-12T20:55:37.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213592025-11-12T20:55:37.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213602025-11-12T20:55:37.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
213612025-11-12T20:55:37.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213622025-11-12T20:55:37.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213632025-11-12T20:55:37.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213642025-11-12T20:55:37.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213652025-11-12T20:55:37.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213662025-11-12T20:55:37.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213672025-11-12T20:55:37.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213682025-11-12T20:55:37.116ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
213692025-11-12T20:55:37.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213702025-11-12T20:55:37.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213712025-11-12T20:55:37.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213722025-11-12T20:55:37.120ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
213732025-11-12T20:55:37.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213742025-11-12T20:55:37.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213752025-11-12T20:55:37.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213762025-11-12T20:55:37.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213772025-11-12T20:55:37.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213782025-11-12T20:55:37.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213792025-11-12T20:55:37.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213802025-11-12T20:55:37.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
213812025-11-12T20:55:37.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213822025-11-12T20:55:37.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213832025-11-12T20:55:37.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213842025-11-12T20:55:37.132ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
213852025-11-12T20:55:37.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213862025-11-12T20:55:37.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213872025-11-12T20:55:37.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213882025-11-12T20:55:37.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213892025-11-12T20:55:37.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213902025-11-12T20:55:37.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213912025-11-12T20:55:37.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213922025-11-12T20:55:37.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
213932025-11-12T20:55:37.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213942025-11-12T20:55:37.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213952025-11-12T20:55:37.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213962025-11-12T20:55:37.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213972025-11-12T20:55:37.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213982025-11-12T20:55:37.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
213992025-11-12T20:55:37.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214002025-11-12T20:55:37.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214012025-11-12T20:55:37.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214022025-11-12T20:55:37.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214032025-11-12T20:55:37.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214042025-11-12T20:55:37.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
214052025-11-12T20:55:37.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214062025-11-12T20:55:37.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214072025-11-12T20:55:37.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214082025-11-12T20:55:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214092025-11-12T20:55:37.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214102025-11-12T20:55:37.159ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = da575366-ca7a-460e-9ae6-b9457f2d2443 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
214112025-11-12T20:55:37.159ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
214122025-11-12T20:55:37.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
214132025-11-12T20:55:37.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
214142025-11-12T20:55:37.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214152025-11-12T20:55:37.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214162025-11-12T20:55:37.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214172025-11-12T20:55:37.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214182025-11-12T20:55:37.163ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(d09e62cf-5276-4e20-ace2-c56774b2adea, 0), (8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0, 0), (fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51, 0)]
214192025-11-12T20:55:37.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214202025-11-12T20:55:37.164ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 661209181 } })
214212025-11-12T20:55:37.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
214222025-11-12T20:55:37.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214232025-11-12T20:55:37.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214242025-11-12T20:55:37.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214252025-11-12T20:55:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214262025-11-12T20:55:37.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214272025-11-12T20:55:37.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
214282025-11-12T20:55:37.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214292025-11-12T20:55:37.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214302025-11-12T20:55:37.175ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
214312025-11-12T20:55:37.175ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
214322025-11-12T20:55:37.176ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = b867eee5-6ce7-46b2-970e-b5662b97d4ce parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
214332025-11-12T20:55:37.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214342025-11-12T20:55:37.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214352025-11-12T20:55:37.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214362025-11-12T20:55:37.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
214372025-11-12T20:55:37.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214382025-11-12T20:55:37.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214392025-11-12T20:55:37.184ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
214402025-11-12T20:55:37.184ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
214412025-11-12T20:55:37.184ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 228.624007ms iteration = 26 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
214422025-11-12T20:55:37.200ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 696721670 } })
214432025-11-12T20:55:37.200ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
214442025-11-12T20:55:37.200ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
214452025-11-12T20:55:37.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
214462025-11-12T20:55:37.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "58e9014e-3f09-4cca-9b94-cd3d9fafd1f7", "content-length": "476", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214472025-11-12T20:55:37.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
214482025-11-12T20:55:37.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fe5fa6c-386d-4834-9872-41b1828097e2", "content-length": "734", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214492025-11-12T20:55:37.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
214502025-11-12T20:55:37.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b141c04-6364-4ab2-aa46-cf1d319ad3ea", "content-length": "111", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214512025-11-12T20:55:37.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
214522025-11-12T20:55:37.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46e1bac8-d074-4fd7-bf7d-ac48089fe5eb", "content-length": "111", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214532025-11-12T20:55:37.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
214542025-11-12T20:55:37.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6250f579-2dad-44d4-bec2-1c445d530e1c", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214552025-11-12T20:55:37.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
214562025-11-12T20:55:37.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "89143432-b070-441a-9909-ca39efc20d12", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214572025-11-12T20:55:37.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
214582025-11-12T20:55:37.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fa9447a-5e47-4f9c-9f39-89fdef7be2c2", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214592025-11-12T20:55:37.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
214602025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "185f198d-e0eb-4b67-bfe2-a52f7686e4b2", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214612025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
214622025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88d1c9c1-f15f-46a8-b027-1dad994eb274", "content-length": "702", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214632025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
214642025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb8add6d-4917-4351-aae7-e7a6f728e3d7", "content-length": "727", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214652025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
214662025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66abfce2-14fa-427f-9519-1e78067d676a", "content-length": "729", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214672025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
214682025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11a4b8b0-6219-43b4-89e9-c299c002caf1", "content-length": "728", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214692025-11-12T20:55:37.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
214702025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5b686e1-cce1-4dc3-bbb8-747fc7424634", "content-length": "722", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214712025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
214722025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9af0c72-929c-43a6-84db-e43e96ece146", "content-length": "10", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214732025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
214742025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d83895b1-f910-4073-a7d8-70e49c658f96", "content-length": "144", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214752025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
214762025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee357233-6135-49e1-8c36-b3332985ea95", "content-length": "147", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214772025-11-12T20:55:37.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
214782025-11-12T20:55:37.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2707103d-5e2b-41b4-94ca-2c184c887fa6", "content-length": "109", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214792025-11-12T20:55:37.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
214802025-11-12T20:55:37.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd84903d-50ad-4036-8473-133bffe99329", "content-length": "109", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214812025-11-12T20:55:37.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
214822025-11-12T20:55:37.207ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f432229e-6e31-45e0-a8b5-41793044be6a", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214832025-11-12T20:55:37.207ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
214842025-11-12T20:55:37.207ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d3d53a9-b5d5-4644-90ab-963d4ff0950b", "content-length": "163", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214852025-11-12T20:55:37.207ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
214862025-11-12T20:55:37.208ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05d34fa8-79c8-4064-bb92-6367786e69af", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214872025-11-12T20:55:37.208ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
214882025-11-12T20:55:37.208ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "879e14cb-5ff8-42ad-ad1e-00f3b775011e", "content-length": "166", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214892025-11-12T20:55:37.208ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
214902025-11-12T20:55:37.208ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ad245b0-7335-4631-b0bc-f32cd7d2227c", "content-length": "702", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214912025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
214922025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3bec341-6270-4cbb-9e16-c991f8a6378b", "content-length": "727", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214932025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
214942025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ee21a7e-01d7-4895-bbdf-5d8cd03d20b7", "content-length": "729", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214952025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
214962025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3e276c3-6fbf-4fec-9fb2-d531431a6fe2", "content-length": "728", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
214972025-11-12T20:55:37.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
214982025-11-12T20:55:37.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
214992025-11-12T20:55:37.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215002025-11-12T20:55:37.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
215012025-11-12T20:55:37.214ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
215022025-11-12T20:55:37.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
215032025-11-12T20:55:37.216ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215042025-11-12T20:55:37.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
215052025-11-12T20:55:37.218ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
215062025-11-12T20:55:37.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
215072025-11-12T20:55:37.252ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
215082025-11-12T20:55:37.252ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
215092025-11-12T20:55:37.310ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "394ed814-772e-43ef-ae59-05f805eece57", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215102025-11-12T20:55:37.312ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
215112025-11-12T20:55:37.312ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
215122025-11-12T20:55:37.312ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
215132025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
215142025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
215152025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
215162025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
215172025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
215182025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "de110472-7f25-4312-966a-87a973fca329", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215192025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
215202025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "04aa8464-b6e2-4368-82fb-38456735e2ef", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215212025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "887a73c3-f04d-464d-87cd-55ceb005b498", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215222025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "baa4fef1-0155-4cd5-a9c3-92648959972f", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215232025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
215242025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
215252025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
215262025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ce2c0f8-b051-44bb-8d8f-df19b8cbd0f4", "content-length": "64", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215272025-11-12T20:55:37.312ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
215282025-11-12T20:55:37.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "0edb38e3-0283-4769-a217-5f4080c67212", "content-length": "416454", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215292025-11-12T20:55:37.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "65fbed50-cce2-418a-a537-e3eae2345511", "content-length": "406795", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215302025-11-12T20:55:37.354ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "02596d8e-a1a9-411e-acb6-01f52546b1b2", "content-length": "435804", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215312025-11-12T20:55:37.354ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ebbe16-5796-4ae6-ab04-af9b1cbf5470", "content-length": "411936", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215322025-11-12T20:55:37.354ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a1c172-0875-4340-a6f5-477c05b9d4f8", "content-length": "408150", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215332025-11-12T20:55:37.358ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
215342025-11-12T20:55:37.358ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
215352025-11-12T20:55:37.358ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
215362025-11-12T20:55:37.358ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
215372025-11-12T20:55:37.358ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
215382025-11-12T20:55:37.393ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
215392025-11-12T20:55:37.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
215402025-11-12T20:55:37.398ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "b44c1466-3092-49e2-9c63-538c39b97764", "content-length": "127", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215412025-11-12T20:55:37.400ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
215422025-11-12T20:55:37.490ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
215432025-11-12T20:55:37.490ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
215442025-11-12T20:55:37.490ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4794c0d-50ac-4748-ba5d-73be74f93672", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215452025-11-12T20:55:37.490ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
215462025-11-12T20:55:37.492ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
215472025-11-12T20:55:37.492ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
215482025-11-12T20:55:37.492ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "66590dbf-7ead-422b-afed-c8ade9c226be", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215492025-11-12T20:55:37.492ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
215502025-11-12T20:55:37.493ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
215512025-11-12T20:55:37.493ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
215522025-11-12T20:55:37.494ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a40d512a-36b9-4ef9-b52d-1a260a723d74", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:55:37 GMT"} })
215532025-11-12T20:55:37.494ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
215542025-11-12T20:55:37.495ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
215552025-11-12T20:55:37.495ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
215562025-11-12T20:55:37.497ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
215572025-11-12T20:55:37.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
215582025-11-12T20:55:37.499ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215592025-11-12T20:55:37.501ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
215602025-11-12T20:55:37.501ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
215612025-11-12T20:55:37.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
215622025-11-12T20:55:37.585ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 7a3a620a-d640-4ac9-9394-c0f9ac495599 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
215632025-11-12T20:55:37.585ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 7a3a620a-d640-4ac9-9394-c0f9ac495599 time_started = 2025-11-12 20:55:37.200613 UTC
215642025-11-12T20:55:37.585ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 54 reason = Dependency
215652025-11-12T20:55:37.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
215662025-11-12T20:55:37.585ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 786.435241ms iteration = 13 status = {"collection_id":"7a3a620a-d640-4ac9-9394-c0f9ac495599","time_done":"2025-11-12 20:55:37.495824 UTC","time_started":"2025-11-12 20:55:37.200613 UTC"}
215672025-11-12T20:55:37.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
215682025-11-12T20:55:37.588ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215692025-11-12T20:55:37.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
215702025-11-12T20:55:37.591ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
215712025-11-12T20:55:37.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
215722025-11-12T20:55:37.747ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
215732025-11-12T20:55:37.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
215742025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 17 reason = Dependency
215752025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 201.086094ms iteration = 54 status = {"Loaded":{"collection_id":"229ed602-3638-48e4-9414-366c12eb36a9","time_loaded":"2025-11-12T20:55:37.585079389Z","time_started":"2025-11-12T20:55:37.206134Z"}}
215762025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Dependency
215772025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 27 reason = Dependency
215782025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
215792025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
215802025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
215812025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
215822025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
215832025-11-12T20:55:37.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
215842025-11-12T20:55:37.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215852025-11-12T20:55:37.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215862025-11-12T20:55:37.788ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215872025-11-12T20:55:37.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
215882025-11-12T20:55:37.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
215892025-11-12T20:55:37.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
215902025-11-12T20:55:37.791ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
215912025-11-12T20:55:37.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
215922025-11-12T20:55:37.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
215932025-11-12T20:55:37.791ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
215942025-11-12T20:55:37.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
215952025-11-12T20:55:37.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
215962025-11-12T20:55:37.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215972025-11-12T20:55:37.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215982025-11-12T20:55:37.793ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
215992025-11-12T20:55:37.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
216002025-11-12T20:55:37.794ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216012025-11-12T20:55:37.795ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
216022025-11-12T20:55:37.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216032025-11-12T20:55:37.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216042025-11-12T20:55:37.795ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
216052025-11-12T20:55:37.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216062025-11-12T20:55:37.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216072025-11-12T20:55:37.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216082025-11-12T20:55:37.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
216092025-11-12T20:55:37.797ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
216102025-11-12T20:55:37.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216112025-11-12T20:55:37.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
216122025-11-12T20:55:37.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216132025-11-12T20:55:37.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216142025-11-12T20:55:37.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216152025-11-12T20:55:37.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216162025-11-12T20:55:37.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
216172025-11-12T20:55:37.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216182025-11-12T20:55:37.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216192025-11-12T20:55:37.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
216202025-11-12T20:55:37.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216212025-11-12T20:55:37.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216222025-11-12T20:55:37.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216232025-11-12T20:55:37.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216242025-11-12T20:55:37.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
216252025-11-12T20:55:37.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216262025-11-12T20:55:37.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216272025-11-12T20:55:37.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
216282025-11-12T20:55:37.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216292025-11-12T20:55:37.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216302025-11-12T20:55:37.806ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.234093ms iteration = 35 status = {"physical_disks_added":0}
216312025-11-12T20:55:37.807ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
216322025-11-12T20:55:37.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216332025-11-12T20:55:37.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216342025-11-12T20:55:37.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216352025-11-12T20:55:37.812ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
216362025-11-12T20:55:37.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216372025-11-12T20:55:37.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216382025-11-12T20:55:37.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216392025-11-12T20:55:37.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
216402025-11-12T20:55:37.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
216412025-11-12T20:55:37.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
216422025-11-12T20:55:37.817ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
216432025-11-12T20:55:37.817ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 31.721661ms iteration = 17 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"229ed602-3638-48e4-9414-366c12eb36a9","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
216442025-11-12T20:55:37.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216452025-11-12T20:55:37.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216462025-11-12T20:55:37.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216472025-11-12T20:55:37.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216482025-11-12T20:55:37.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216492025-11-12T20:55:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216502025-11-12T20:55:37.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216512025-11-12T20:55:37.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216522025-11-12T20:55:37.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216532025-11-12T20:55:37.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216542025-11-12T20:55:37.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216552025-11-12T20:55:37.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
216562025-11-12T20:55:37.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216572025-11-12T20:55:37.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216582025-11-12T20:55:37.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216592025-11-12T20:55:37.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216602025-11-12T20:55:37.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216612025-11-12T20:55:37.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216622025-11-12T20:55:37.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216632025-11-12T20:55:37.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216642025-11-12T20:55:37.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216652025-11-12T20:55:37.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
216662025-11-12T20:55:37.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216672025-11-12T20:55:37.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216682025-11-12T20:55:37.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216692025-11-12T20:55:37.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216702025-11-12T20:55:37.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216712025-11-12T20:55:37.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
216722025-11-12T20:55:37.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216732025-11-12T20:55:37.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216742025-11-12T20:55:37.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216752025-11-12T20:55:37.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216762025-11-12T20:55:37.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216772025-11-12T20:55:37.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
216782025-11-12T20:55:37.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216792025-11-12T20:55:37.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216802025-11-12T20:55:37.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216812025-11-12T20:55:37.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216822025-11-12T20:55:37.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216832025-11-12T20:55:37.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216842025-11-12T20:55:37.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216852025-11-12T20:55:37.904ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216862025-11-12T20:55:37.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216872025-11-12T20:55:37.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216882025-11-12T20:55:37.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216892025-11-12T20:55:37.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216902025-11-12T20:55:37.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216912025-11-12T20:55:37.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216922025-11-12T20:55:37.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216932025-11-12T20:55:37.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216942025-11-12T20:55:37.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216952025-11-12T20:55:37.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216962025-11-12T20:55:37.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216972025-11-12T20:55:37.918ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216982025-11-12T20:55:37.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216992025-11-12T20:55:37.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217002025-11-12T20:55:37.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217012025-11-12T20:55:37.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
217022025-11-12T20:55:37.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217032025-11-12T20:55:37.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217042025-11-12T20:55:37.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217052025-11-12T20:55:37.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217062025-11-12T20:55:37.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217072025-11-12T20:55:37.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217082025-11-12T20:55:37.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217092025-11-12T20:55:37.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
217102025-11-12T20:55:37.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217112025-11-12T20:55:37.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217122025-11-12T20:55:37.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217132025-11-12T20:55:37.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217142025-11-12T20:55:37.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217152025-11-12T20:55:37.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 298 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = e7bc4719-4bee-4f3f-a97f-a018c963b9b4 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
217162025-11-12T20:55:37.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
217172025-11-12T20:55:37.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217182025-11-12T20:55:37.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217192025-11-12T20:55:37.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217202025-11-12T20:55:37.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217212025-11-12T20:55:37.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217222025-11-12T20:55:37.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
217232025-11-12T20:55:37.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217242025-11-12T20:55:37.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217252025-11-12T20:55:37.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217262025-11-12T20:55:37.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217272025-11-12T20:55:37.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217282025-11-12T20:55:37.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
217292025-11-12T20:55:37.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217302025-11-12T20:55:37.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217312025-11-12T20:55:37.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217322025-11-12T20:55:37.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217332025-11-12T20:55:37.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217342025-11-12T20:55:37.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
217352025-11-12T20:55:37.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217362025-11-12T20:55:37.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217372025-11-12T20:55:37.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217382025-11-12T20:55:37.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217392025-11-12T20:55:37.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217402025-11-12T20:55:37.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
217412025-11-12T20:55:37.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217422025-11-12T20:55:37.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217432025-11-12T20:55:37.980ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
217442025-11-12T20:55:37.980ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
217452025-11-12T20:55:37.980ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 545c5520-e86a-4b1d-b38a-9a292cdc7e50 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
217462025-11-12T20:55:37.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217472025-11-12T20:55:37.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217482025-11-12T20:55:37.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217492025-11-12T20:55:37.986ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
217502025-11-12T20:55:37.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217512025-11-12T20:55:37.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217522025-11-12T20:55:37.990ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
217532025-11-12T20:55:37.990ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
217542025-11-12T20:55:37.990ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 204.239755ms iteration = 27 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
217552025-11-12T20:55:40.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 122 reason = Timeout
217562025-11-12T20:55:40.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
217572025-11-12T20:55:40.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
217582025-11-12T20:55:40.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217592025-11-12T20:55:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
217602025-11-12T20:55:40.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
217612025-11-12T20:55:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
217622025-11-12T20:55:40.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
217632025-11-12T20:55:40.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.325244ms iteration = 122 status = {"config_updated":false}
217642025-11-12T20:55:45.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
217652025-11-12T20:55:45.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
217662025-11-12T20:55:45.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
217672025-11-12T20:55:45.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 123 reason = Timeout
217682025-11-12T20:55:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
217692025-11-12T20:55:45.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
217702025-11-12T20:55:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
217712025-11-12T20:55:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
217722025-11-12T20:55:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
217732025-11-12T20:55:45.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217742025-11-12T20:55:45.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
217752025-11-12T20:55:45.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217762025-11-12T20:55:45.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
217772025-11-12T20:55:45.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.465186ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
217782025-11-12T20:55:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
217792025-11-12T20:55:45.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
217802025-11-12T20:55:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
217812025-11-12T20:55:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
217822025-11-12T20:55:45.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
217832025-11-12T20:55:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
217842025-11-12T20:55:45.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
217852025-11-12T20:55:45.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.908709ms iteration = 123 status = {"config_updated":false}
217862025-11-12T20:55:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
217872025-11-12T20:55:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
217882025-11-12T20:55:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217892025-11-12T20:55:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
217902025-11-12T20:55:45.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
217912025-11-12T20:55:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
217922025-11-12T20:55:45.302ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
217932025-11-12T20:55:45.302ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.604887ms iteration = 62 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
217942025-11-12T20:55:47.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 389 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = eda1f92b-b9e3-48ed-ada8-b53c17375216 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
217952025-11-12T20:55:50.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 55 reason = Timeout
217962025-11-12T20:55:50.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
217972025-11-12T20:55:50.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
217982025-11-12T20:55:50.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 124 reason = Timeout
217992025-11-12T20:55:50.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218002025-11-12T20:55:50.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
218012025-11-12T20:55:50.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218022025-11-12T20:55:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218032025-11-12T20:55:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
218042025-11-12T20:55:50.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
218052025-11-12T20:55:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
218062025-11-12T20:55:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
218072025-11-12T20:55:50.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218082025-11-12T20:55:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
218092025-11-12T20:55:50.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 42 reason = Timeout
218102025-11-12T20:55:50.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
218112025-11-12T20:55:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
218122025-11-12T20:55:50.284ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 229ed602-3638-48e4-9414-366c12eb36a9 original_time_started = 2025-11-12 20:55:37.206134 UTC
218132025-11-12T20:55:50.284ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.699059ms iteration = 55 status = {"Loaded":{"collection_id":"229ed602-3638-48e4-9414-366c12eb36a9","time_loaded":"2025-11-12T20:55:50.276975474Z","time_started":"2025-11-12T20:55:37.206134Z"}}
218142025-11-12T20:55:50.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
218152025-11-12T20:55:50.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.72047ms iteration = 124 status = {"config_updated":false}
218162025-11-12T20:55:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218172025-11-12T20:55:50.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
218182025-11-12T20:55:50.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
218192025-11-12T20:55:50.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
218202025-11-12T20:55:50.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.558154ms iteration = 42 status = "NoSitrep"
218212025-11-12T20:55:50.291ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 42 reason = Timeout
218222025-11-12T20:55:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
218232025-11-12T20:55:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
218242025-11-12T20:55:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
218252025-11-12T20:55:50.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
218262025-11-12T20:55:50.293ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218272025-11-12T20:55:50.294ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
218282025-11-12T20:55:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
218292025-11-12T20:55:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
218302025-11-12T20:55:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
218312025-11-12T20:55:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
218322025-11-12T20:55:50.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218332025-11-12T20:55:50.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
218342025-11-12T20:55:50.301ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
218352025-11-12T20:55:50.301ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.03096ms iteration = 42 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
218362025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
218372025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
218382025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
218392025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 125 reason = Timeout
218402025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
218412025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218422025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
218432025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
218442025-11-12T20:55:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
218452025-11-12T20:55:55.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218462025-11-12T20:55:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
218472025-11-12T20:55:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218482025-11-12T20:55:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
218492025-11-12T20:55:55.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.933932ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
218502025-11-12T20:55:55.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
218512025-11-12T20:55:55.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218522025-11-12T20:55:55.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
218532025-11-12T20:55:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
218542025-11-12T20:55:55.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
218552025-11-12T20:55:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
218562025-11-12T20:55:55.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
218572025-11-12T20:55:55.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.457913ms iteration = 125 status = {"config_updated":false}
218582025-11-12T20:55:55.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
218592025-11-12T20:55:55.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
218602025-11-12T20:55:55.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218612025-11-12T20:55:55.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
218622025-11-12T20:55:55.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
218632025-11-12T20:55:55.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
218642025-11-12T20:55:55.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
218652025-11-12T20:55:55.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.098483ms iteration = 63 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
218662025-11-12T20:55:57.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 292 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 12a1f78b-03c3-4902-92d1-549879156200 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
218672025-11-12T20:56:00.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 126 reason = Timeout
218682025-11-12T20:56:00.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218692025-11-12T20:56:00.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
218702025-11-12T20:56:00.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218712025-11-12T20:56:00.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
218722025-11-12T20:56:00.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218732025-11-12T20:56:00.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
218742025-11-12T20:56:00.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
218752025-11-12T20:56:00.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.894478ms iteration = 126 status = {"config_updated":false}
218762025-11-12T20:56:05.274ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
218772025-11-12T20:56:05.274ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
218782025-11-12T20:56:05.274ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
218792025-11-12T20:56:05.274ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
218802025-11-12T20:56:05.274ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
218812025-11-12T20:56:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
218822025-11-12T20:56:05.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
218832025-11-12T20:56:05.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 56 reason = Timeout
218842025-11-12T20:56:05.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
218852025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
218862025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
218872025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
218882025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
218892025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
218902025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
218912025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
218922025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
218932025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
218942025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
218952025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
218962025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
218972025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 127 reason = Timeout
218982025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
218992025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
219002025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 33 reason = Timeout
219012025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
219022025-11-12T20:56:05.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
219032025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
219042025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
219052025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
219062025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
219072025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
219082025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
219092025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
219102025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
219112025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
219122025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
219132025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
219142025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
219152025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
219162025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
219172025-11-12T20:56:05.277ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1560, tv_nsec: 774349279 } })
219182025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
219192025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
219202025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
219212025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
219222025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
219232025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
219242025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
219252025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
219262025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
219272025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
219282025-11-12T20:56:05.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1560, tv_nsec: 774723309 } })
219292025-11-12T20:56:05.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1560, tv_nsec: 774733979 } })
219302025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
219312025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
219322025-11-12T20:56:05.277ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
219332025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
219342025-11-12T20:56:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
219352025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
219362025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
219372025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
219382025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
219392025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
219402025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
219412025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
219422025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
219432025-11-12T20:56:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
219442025-11-12T20:56:05.279ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.336178ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
219452025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219462025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
219472025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219482025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219492025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219502025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219512025-11-12T20:56:05.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219522025-11-12T20:56:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
219532025-11-12T20:56:05.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219542025-11-12T20:56:05.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.215929ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
219552025-11-12T20:56:05.282ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
219562025-11-12T20:56:05.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
219572025-11-12T20:56:05.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 43 reason = Timeout
219582025-11-12T20:56:05.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
219592025-11-12T20:56:05.284ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219602025-11-12T20:56:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
219612025-11-12T20:56:05.285ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219622025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
219632025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
219642025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.271996ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
219652025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
219662025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
219672025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
219682025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
219692025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
219702025-11-12T20:56:05.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 10.772529ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
219712025-11-12T20:56:05.288ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
219722025-11-12T20:56:05.288ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
219732025-11-12T20:56:05.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
219742025-11-12T20:56:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
219752025-11-12T20:56:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
219762025-11-12T20:56:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
219772025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
219782025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
219792025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
219802025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
219812025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
219822025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
219832025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
219842025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
219852025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
219862025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
219872025-11-12T20:56:05.289ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
219882025-11-12T20:56:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
219892025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 43 reason = Timeout
219902025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
219912025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219922025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
219932025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219942025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
219952025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
219962025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
219972025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
219982025-11-12T20:56:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
219992025-11-12T20:56:05.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
220002025-11-12T20:56:05.298ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
220012025-11-12T20:56:05.298ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 21.28844ms iteration = 127 status = {"config_updated":false}
220022025-11-12T20:56:05.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
220032025-11-12T20:56:05.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
220042025-11-12T20:56:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.446001ms iteration = 22 status = 0
220052025-11-12T20:56:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
220062025-11-12T20:56:05.300ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
220072025-11-12T20:56:05.300ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.065307ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
220082025-11-12T20:56:05.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
220092025-11-12T20:56:05.300ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 229ed602-3638-48e4-9414-366c12eb36a9 original_time_started = 2025-11-12 20:55:37.206134 UTC
220102025-11-12T20:56:05.300ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 24.135235ms iteration = 56 status = {"Loaded":{"collection_id":"229ed602-3638-48e4-9414-366c12eb36a9","time_loaded":"2025-11-12T20:56:05.276716374Z","time_started":"2025-11-12T20:55:37.206134Z"}}
220112025-11-12T20:56:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
220122025-11-12T20:56:05.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
220132025-11-12T20:56:05.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
220142025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
220152025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
220162025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
220172025-11-12T20:56:05.302ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
220182025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
220192025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
220202025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
220212025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
220222025-11-12T20:56:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
220232025-11-12T20:56:05.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220242025-11-12T20:56:05.303ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1560, tv_nsec: 800040994 } })
220252025-11-12T20:56:05.303ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
220262025-11-12T20:56:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 20.353736ms iteration = 43 status = "NoSitrep"
220272025-11-12T20:56:05.305ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220282025-11-12T20:56:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220292025-11-12T20:56:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
220302025-11-12T20:56:05.306ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
220312025-11-12T20:56:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
220322025-11-12T20:56:05.311ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
220332025-11-12T20:56:05.311ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
220342025-11-12T20:56:05.311ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 34.658287ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
220352025-11-12T20:56:05.317ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220362025-11-12T20:56:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220372025-11-12T20:56:05.320ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220382025-11-12T20:56:05.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
220392025-11-12T20:56:05.324ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220402025-11-12T20:56:05.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220412025-11-12T20:56:05.328ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220422025-11-12T20:56:05.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
220432025-11-12T20:56:05.331ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220442025-11-12T20:56:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220452025-11-12T20:56:05.346ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
220462025-11-12T20:56:05.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
220472025-11-12T20:56:05.350ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
220482025-11-12T20:56:05.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
220492025-11-12T20:56:05.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.11411ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
220502025-11-12T20:56:05.360ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
220512025-11-12T20:56:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
220522025-11-12T20:56:05.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
220532025-11-12T20:56:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
220542025-11-12T20:56:05.366ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
220552025-11-12T20:56:05.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
220562025-11-12T20:56:05.367ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220572025-11-12T20:56:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
220582025-11-12T20:56:05.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
220592025-11-12T20:56:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
220602025-11-12T20:56:05.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220612025-11-12T20:56:05.378ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220622025-11-12T20:56:05.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
220632025-11-12T20:56:05.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
220642025-11-12T20:56:05.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
220652025-11-12T20:56:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
220662025-11-12T20:56:05.383ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
220672025-11-12T20:56:05.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
220682025-11-12T20:56:05.384ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
220692025-11-12T20:56:05.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
220702025-11-12T20:56:05.385ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 108.005375ms iteration = 36 status = {"physical_disks_added":0}
220712025-11-12T20:56:05.386ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220722025-11-12T20:56:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
220732025-11-12T20:56:05.390ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
220742025-11-12T20:56:05.390ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 99.103614ms iteration = 43 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
220752025-11-12T20:56:05.405ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
220762025-11-12T20:56:05.405ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 128.206248ms iteration = 64 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
220772025-11-12T20:56:05.410ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 133.312079ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
220782025-11-12T20:56:05.412ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 134.765077ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
220792025-11-12T20:56:05.415ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.776796ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
220802025-11-12T20:56:05.425ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
220812025-11-12T20:56:05.427ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 152.893207ms iteration = 22 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
220822025-11-12T20:56:05.428ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
220832025-11-12T20:56:05.428ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 150.973837ms iteration = 33 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
220842025-11-12T20:56:05.428ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
220852025-11-12T20:56:05.428ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0324ce9-1bbc-4982-a838-2dc776ff4e51", "content-length": "78", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
220862025-11-12T20:56:05.429ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
220872025-11-12T20:56:05.429ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
220882025-11-12T20:56:05.429ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
220892025-11-12T20:56:05.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
220902025-11-12T20:56:05.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
220912025-11-12T20:56:05.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
220922025-11-12T20:56:05.431ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
220932025-11-12T20:56:05.431ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
220942025-11-12T20:56:05.432ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
220952025-11-12T20:56:05.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
220962025-11-12T20:56:05.435ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
220972025-11-12T20:56:05.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
220982025-11-12T20:56:05.435ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
220992025-11-12T20:56:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
221002025-11-12T20:56:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
221012025-11-12T20:56:05.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
221022025-11-12T20:56:05.436ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
221032025-11-12T20:56:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
221042025-11-12T20:56:05.438ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
221052025-11-12T20:56:05.438ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
221062025-11-12T20:56:05.438ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "714abb1e-f1cb-4202-834f-33d3065cc4cc", "content-length": "81", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221072025-11-12T20:56:05.473ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
221082025-11-12T20:56:05.473ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
221092025-11-12T20:56:05.474ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9858488-c4d2-406c-a098-5d2800f71b83", "content-length": "81", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221102025-11-12T20:56:05.474ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
221112025-11-12T20:56:05.474ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
221122025-11-12T20:56:05.474ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd01eb00-c751-4230-9483-b365c14dabe5", "content-length": "26", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221132025-11-12T20:56:05.474ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
221142025-11-12T20:56:05.476ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
221152025-11-12T20:56:05.476ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
221162025-11-12T20:56:05.477ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cbfcb23-c146-4a67-949c-f6e389e1e1db", "content-length": "81", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221172025-11-12T20:56:05.477ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
221182025-11-12T20:56:05.479ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 202.011318ms iteration = 22 status = {"disabled":false,"errors":[],"sps":[]}
221192025-11-12T20:56:05.502ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
221202025-11-12T20:56:05.502ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
221212025-11-12T20:56:05.502ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d21043d4-a90f-43c1-b2a1-115ea1d88c12", "content-length": "26", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221222025-11-12T20:56:05.502ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
221232025-11-12T20:56:05.504ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
221242025-11-12T20:56:05.531ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
221252025-11-12T20:56:05.531ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
221262025-11-12T20:56:05.531ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eaffff3-7759-4f2e-9bfa-969e0f4fb7ea", "content-length": "372", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221272025-11-12T20:56:05.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
221282025-11-12T20:56:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
221292025-11-12T20:56:05.544ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
221302025-11-12T20:56:05.544ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
221312025-11-12T20:56:05.544ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b14ec0e6-4790-4a6b-ac66-8098c50a268f", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221322025-11-12T20:56:05.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
221332025-11-12T20:56:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
221342025-11-12T20:56:05.547ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
221352025-11-12T20:56:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
221362025-11-12T20:56:05.550ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
221372025-11-12T20:56:05.550ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
221382025-11-12T20:56:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
221392025-11-12T20:56:05.551ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "62e632e5-2610-4705-866b-73d3fbdecb6f", "content-length": "691", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221402025-11-12T20:56:05.551ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
221412025-11-12T20:56:05.551ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
221422025-11-12T20:56:05.551ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
221432025-11-12T20:56:05.553ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 275.812158ms iteration = 22 status = {}
221442025-11-12T20:56:05.554ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
221452025-11-12T20:56:05.554ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
221462025-11-12T20:56:05.554ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "24d68a6c-6631-46bf-8ca8-73d6972074e9", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221472025-11-12T20:56:05.556ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 279.634727ms iteration = 33 status = {}
221482025-11-12T20:56:05.560ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 57372344 } })
221492025-11-12T20:56:05.570ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 296.35715ms iteration = 22 status = {}
221502025-11-12T20:56:05.624ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221512025-11-12T20:56:05.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221522025-11-12T20:56:05.626ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
221532025-11-12T20:56:05.626ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221542025-11-12T20:56:05.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221552025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
221562025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
221572025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
221582025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
221592025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
221602025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
221612025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
221622025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
221632025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
221642025-11-12T20:56:05.638ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
221652025-11-12T20:56:05.639ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
221662025-11-12T20:56:05.639ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221672025-11-12T20:56:05.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221682025-11-12T20:56:05.641ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
221692025-11-12T20:56:05.641ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
221702025-11-12T20:56:05.747ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "91dac7bb-000e-4873-8580-9228903976db", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221712025-11-12T20:56:05.747ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
221722025-11-12T20:56:05.747ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
221732025-11-12T20:56:05.753ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9c44309-369b-43aa-8c32-c597105804fa", "content-length": "682", "date": "Wed, 12 Nov 2025 20:56:05 GMT"} })
221742025-11-12T20:56:05.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221752025-11-12T20:56:05.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221762025-11-12T20:56:05.757ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221772025-11-12T20:56:05.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221782025-11-12T20:56:05.760ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221792025-11-12T20:56:05.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221802025-11-12T20:56:05.762ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
221812025-11-12T20:56:05.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
221822025-11-12T20:56:05.770ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 493.243595ms iteration = 22 status = {}
221832025-11-12T20:56:07.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 479 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 0402e562-d7e1-4ce0-926b-adaac9865625 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
221842025-11-12T20:56:10.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 128 reason = Timeout
221852025-11-12T20:56:10.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
221862025-11-12T20:56:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
221872025-11-12T20:56:10.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221882025-11-12T20:56:10.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
221892025-11-12T20:56:10.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
221902025-11-12T20:56:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
221912025-11-12T20:56:10.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
221922025-11-12T20:56:10.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.527239ms iteration = 128 status = {"config_updated":false}
221932025-11-12T20:56:15.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
221942025-11-12T20:56:15.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
221952025-11-12T20:56:15.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
221962025-11-12T20:56:15.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
221972025-11-12T20:56:15.278ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 129 reason = Timeout
221982025-11-12T20:56:15.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
221992025-11-12T20:56:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222002025-11-12T20:56:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
222012025-11-12T20:56:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
222022025-11-12T20:56:15.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222032025-11-12T20:56:15.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222042025-11-12T20:56:15.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
222052025-11-12T20:56:15.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
222062025-11-12T20:56:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
222072025-11-12T20:56:15.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
222082025-11-12T20:56:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
222092025-11-12T20:56:15.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222102025-11-12T20:56:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
222112025-11-12T20:56:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222122025-11-12T20:56:15.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
222132025-11-12T20:56:15.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.804002ms iteration = 129 status = {"config_updated":false}
222142025-11-12T20:56:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222152025-11-12T20:56:15.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222162025-11-12T20:56:15.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222172025-11-12T20:56:15.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
222182025-11-12T20:56:15.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222192025-11-12T20:56:15.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222202025-11-12T20:56:15.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
222212025-11-12T20:56:15.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.804226ms iteration = 65 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
222222025-11-12T20:56:15.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.234252ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
222232025-11-12T20:56:17.914ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:47746
222242025-11-12T20:56:17.914ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = fdb94816-c06d-43f4-858b-5634f3f9c178 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
222252025-11-12T20:56:17.914ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = fdb94816-c06d-43f4-858b-5634f3f9c178 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
222262025-11-12T20:56:17.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = fdb94816-c06d-43f4-858b-5634f3f9c178 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
222272025-11-12T20:56:17.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3192 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = fdb94816-c06d-43f4-858b-5634f3f9c178 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
222282025-11-12T20:56:17.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 06fddd1e-4d91-4877-8bfc-35aba932faca uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
222292025-11-12T20:56:17.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 06fddd1e-4d91-4877-8bfc-35aba932faca roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
222302025-11-12T20:56:17.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 06fddd1e-4d91-4877-8bfc-35aba932faca resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
222312025-11-12T20:56:17.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2584 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 06fddd1e-4d91-4877-8bfc-35aba932faca response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
222322025-11-12T20:56:17.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 259 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 69fefbe2-c4b8-4da8-a043-f44dcfb79266 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
222332025-11-12T20:56:20.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 57 reason = Timeout
222342025-11-12T20:56:20.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
222352025-11-12T20:56:20.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
222362025-11-12T20:56:20.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 130 reason = Timeout
222372025-11-12T20:56:20.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
222382025-11-12T20:56:20.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
222392025-11-12T20:56:20.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222402025-11-12T20:56:20.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222412025-11-12T20:56:20.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
222422025-11-12T20:56:20.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
222432025-11-12T20:56:20.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
222442025-11-12T20:56:20.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
222452025-11-12T20:56:20.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
222462025-11-12T20:56:20.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
222472025-11-12T20:56:20.282ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 229ed602-3638-48e4-9414-366c12eb36a9 original_time_started = 2025-11-12 20:55:37.206134 UTC
222482025-11-12T20:56:20.282ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 6.147749ms iteration = 57 status = {"Loaded":{"collection_id":"229ed602-3638-48e4-9414-366c12eb36a9","time_loaded":"2025-11-12T20:56:20.276021378Z","time_started":"2025-11-12T20:55:37.206134Z"}}
222492025-11-12T20:56:20.282ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
222502025-11-12T20:56:20.282ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.059147ms iteration = 130 status = {"config_updated":false}
222512025-11-12T20:56:20.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 44 reason = Timeout
222522025-11-12T20:56:20.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
222532025-11-12T20:56:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
222542025-11-12T20:56:20.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222552025-11-12T20:56:20.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
222562025-11-12T20:56:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
222572025-11-12T20:56:20.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
222582025-11-12T20:56:20.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.408199ms iteration = 44 status = "NoSitrep"
222592025-11-12T20:56:20.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 44 reason = Timeout
222602025-11-12T20:56:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
222612025-11-12T20:56:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
222622025-11-12T20:56:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
222632025-11-12T20:56:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
222642025-11-12T20:56:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222652025-11-12T20:56:20.293ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
222662025-11-12T20:56:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
222672025-11-12T20:56:20.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
222682025-11-12T20:56:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
222692025-11-12T20:56:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
222702025-11-12T20:56:20.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222712025-11-12T20:56:20.299ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
222722025-11-12T20:56:20.300ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
222732025-11-12T20:56:20.300ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.839564ms iteration = 44 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
222742025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
222752025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
222762025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
222772025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222782025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 131 reason = Timeout
222792025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
222802025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222812025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
222822025-11-12T20:56:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
222832025-11-12T20:56:25.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222842025-11-12T20:56:25.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222852025-11-12T20:56:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
222862025-11-12T20:56:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222872025-11-12T20:56:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
222882025-11-12T20:56:25.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
222892025-11-12T20:56:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222902025-11-12T20:56:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
222912025-11-12T20:56:25.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222922025-11-12T20:56:25.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
222932025-11-12T20:56:25.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.70228ms iteration = 131 status = {"config_updated":false}
222942025-11-12T20:56:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222952025-11-12T20:56:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222962025-11-12T20:56:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
222972025-11-12T20:56:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
222982025-11-12T20:56:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
222992025-11-12T20:56:25.301ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
223002025-11-12T20:56:25.301ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.916369ms iteration = 66 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
223012025-11-12T20:56:25.312ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
223022025-11-12T20:56:25.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
223032025-11-12T20:56:25.347ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.799106ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
223042025-11-12T20:56:27.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 362 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 656dad72-0d5e-40ff-b9d6-202c46dc2b58 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
223052025-11-12T20:56:30.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 132 reason = Timeout
223062025-11-12T20:56:30.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
223072025-11-12T20:56:30.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
223082025-11-12T20:56:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223092025-11-12T20:56:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
223102025-11-12T20:56:30.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
223112025-11-12T20:56:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
223122025-11-12T20:56:30.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
223132025-11-12T20:56:30.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.750315ms iteration = 132 status = {"config_updated":false}
223142025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
223152025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
223162025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
223172025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
223182025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
223192025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
223202025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
223212025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
223222025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
223232025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
223242025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
223252025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
223262025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
223272025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
223282025-11-12T20:56:35.274ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
223292025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
223302025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
223312025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
223322025-11-12T20:56:35.274ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = ExternalNetworking step id = Ensure
223332025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
223342025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
223352025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223362025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
223372025-11-12T20:56:35.274ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:46:35.274502415 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
223382025-11-12T20:56:35.274ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
223392025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223402025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
223412025-11-12T20:56:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 772426627 } })
223422025-11-12T20:56:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 772450347 } })
223432025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
223442025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
223452025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
223462025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
223472025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
223482025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
223492025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.287533ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
223502025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
223512025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 58 reason = Timeout
223522025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.53327ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
223532025-11-12T20:56:35.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
223542025-11-12T20:56:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
223552025-11-12T20:56:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
223562025-11-12T20:56:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
223572025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
223582025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
223592025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
223602025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
223612025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
223622025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
223632025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 28 reason = Timeout
223642025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223652025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
223662025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
223672025-11-12T20:56:35.277ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
223682025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
223692025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
223702025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
223712025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
223722025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
223732025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
223742025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
223752025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
223762025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
223772025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 34 reason = Timeout
223782025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
223792025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
223802025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
223812025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
223822025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
223832025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
223842025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
223852025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
223862025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
223872025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
223882025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
223892025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
223902025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
223912025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
223922025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
223932025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
223942025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
223952025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
223962025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
223972025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
223982025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
223992025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
224002025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 133 reason = Timeout
224012025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
224022025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
224032025-11-12T20:56:35.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
224042025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Timeout
224052025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
224062025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
224072025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
224082025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
224092025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
224102025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
224112025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
224122025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
224132025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
224142025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
224152025-11-12T20:56:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
224162025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
224172025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224182025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
224192025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
224202025-11-12T20:56:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 775488616 } })
224212025-11-12T20:56:35.278ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 775509576 } })
224222025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
224232025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
224242025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224252025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
224262025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
224272025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
224282025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
224292025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
224302025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
224312025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
224322025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
224332025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
224342025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
224352025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
224362025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
224372025-11-12T20:56:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
224382025-11-12T20:56:35.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224392025-11-12T20:56:35.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 776330968 } })
224402025-11-12T20:56:35.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
224412025-11-12T20:56:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
224422025-11-12T20:56:35.279ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:46:35.274502415 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
224432025-11-12T20:56:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.426071ms iteration = 12 status = {"expiration":"2025-11-12T20:46:35.274502415Z","pruned":[]}
224442025-11-12T20:56:35.280ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224452025-11-12T20:56:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
224462025-11-12T20:56:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224472025-11-12T20:56:35.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224482025-11-12T20:56:35.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224492025-11-12T20:56:35.281ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
224502025-11-12T20:56:35.281ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.071845ms iteration = 12 status = {"by_rx":{},"error":null}
224512025-11-12T20:56:35.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224522025-11-12T20:56:35.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224532025-11-12T20:56:35.282ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.816074ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
224542025-11-12T20:56:35.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224552025-11-12T20:56:35.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 5.297457ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
224562025-11-12T20:56:35.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
224572025-11-12T20:56:35.283ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
224582025-11-12T20:56:35.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
224592025-11-12T20:56:35.283ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224602025-11-12T20:56:35.283ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.474415ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
224612025-11-12T20:56:35.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 45 reason = Timeout
224622025-11-12T20:56:35.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
224632025-11-12T20:56:35.284ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224642025-11-12T20:56:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
224652025-11-12T20:56:35.285ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224662025-11-12T20:56:35.285ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
224672025-11-12T20:56:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
224682025-11-12T20:56:35.285ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
224692025-11-12T20:56:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
224702025-11-12T20:56:35.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
224712025-11-12T20:56:35.286ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.958051ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
224722025-11-12T20:56:35.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224732025-11-12T20:56:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
224742025-11-12T20:56:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
224752025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
224762025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224772025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224782025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224792025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
224802025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
224812025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
224822025-11-12T20:56:35.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224832025-11-12T20:56:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
224842025-11-12T20:56:35.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224852025-11-12T20:56:35.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 45 reason = Timeout
224862025-11-12T20:56:35.289ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 12 reason = Timeout
224872025-11-12T20:56:35.289ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
224882025-11-12T20:56:35.289ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
224892025-11-12T20:56:35.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
224902025-11-12T20:56:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
224912025-11-12T20:56:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
224922025-11-12T20:56:35.290ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 13.280114ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
224932025-11-12T20:56:35.290ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
224942025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
224952025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
224962025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
224972025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
224982025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
224992025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225002025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225012025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
225022025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225032025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225042025-11-12T20:56:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
225052025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
225062025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
225072025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225082025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
225092025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
225102025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
225112025-11-12T20:56:35.292ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
225122025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
225132025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225142025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
225152025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225162025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
225172025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
225182025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
225192025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
225202025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
225212025-11-12T20:56:35.293ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225222025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225232025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
225242025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
225252025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225262025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225272025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
225282025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
225292025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225302025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
225312025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
225322025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
225332025-11-12T20:56:35.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
225342025-11-12T20:56:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
225352025-11-12T20:56:35.296ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 229ed602-3638-48e4-9414-366c12eb36a9 original_time_started = 2025-11-12 20:55:37.206134 UTC
225362025-11-12T20:56:35.296ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 20.579213ms iteration = 58 status = {"Loaded":{"collection_id":"229ed602-3638-48e4-9414-366c12eb36a9","time_loaded":"2025-11-12T20:56:35.275802028Z","time_started":"2025-11-12T20:55:37.206134Z"}}
225372025-11-12T20:56:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
225382025-11-12T20:56:35.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
225392025-11-12T20:56:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
225402025-11-12T20:56:35.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
225412025-11-12T20:56:35.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
225422025-11-12T20:56:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
225432025-11-12T20:56:35.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
225442025-11-12T20:56:35.299ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
225452025-11-12T20:56:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
225462025-11-12T20:56:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
225472025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
225482025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
225492025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
225502025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 23.547079ms iteration = 133 status = {"config_updated":false}
225512025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225522025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
225532025-11-12T20:56:35.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225542025-11-12T20:56:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
225552025-11-12T20:56:35.302ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
225562025-11-12T20:56:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
225572025-11-12T20:56:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
225582025-11-12T20:56:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
225592025-11-12T20:56:35.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
225602025-11-12T20:56:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
225612025-11-12T20:56:35.312ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 28.238981ms iteration = 45 status = "NoSitrep"
225622025-11-12T20:56:35.313ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
225632025-11-12T20:56:35.314ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1590, tv_nsec: 811592550 } })
225642025-11-12T20:56:35.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
225652025-11-12T20:56:35.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225662025-11-12T20:56:35.314ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
225672025-11-12T20:56:35.315ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 37.900921ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
225682025-11-12T20:56:35.319ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
225692025-11-12T20:56:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
225702025-11-12T20:56:35.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225712025-11-12T20:56:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225722025-11-12T20:56:35.321ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 44.214534ms iteration = 23 status = 0
225732025-11-12T20:56:35.322ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225742025-11-12T20:56:35.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225752025-11-12T20:56:35.323ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
225762025-11-12T20:56:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225772025-11-12T20:56:35.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
225782025-11-12T20:56:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
225792025-11-12T20:56:35.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225802025-11-12T20:56:35.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225812025-11-12T20:56:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225822025-11-12T20:56:35.327ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
225832025-11-12T20:56:35.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
225842025-11-12T20:56:35.327ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
225852025-11-12T20:56:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
225862025-11-12T20:56:35.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225872025-11-12T20:56:35.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225882025-11-12T20:56:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
225892025-11-12T20:56:35.331ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
225902025-11-12T20:56:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
225912025-11-12T20:56:35.332ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
225922025-11-12T20:56:35.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
225932025-11-12T20:56:35.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
225942025-11-12T20:56:35.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
225952025-11-12T20:56:35.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
225962025-11-12T20:56:35.333ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
225972025-11-12T20:56:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
225982025-11-12T20:56:35.335ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
225992025-11-12T20:56:35.335ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 45.390585ms iteration = 45 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
226002025-11-12T20:56:35.335ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226012025-11-12T20:56:35.339ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
226022025-11-12T20:56:35.339ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
226032025-11-12T20:56:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
226042025-11-12T20:56:35.340ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
226052025-11-12T20:56:35.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226062025-11-12T20:56:35.346ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
226072025-11-12T20:56:35.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226082025-11-12T20:56:35.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226092025-11-12T20:56:35.348ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
226102025-11-12T20:56:35.353ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
226112025-11-12T20:56:35.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226122025-11-12T20:56:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226132025-11-12T20:56:35.356ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
226142025-11-12T20:56:35.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226152025-11-12T20:56:35.359ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
226162025-11-12T20:56:35.359ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
226172025-11-12T20:56:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226182025-11-12T20:56:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
226192025-11-12T20:56:35.359ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
226202025-11-12T20:56:35.359ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 82.222518ms iteration = 67 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
226212025-11-12T20:56:35.361ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
226222025-11-12T20:56:35.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.22804ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
226232025-11-12T20:56:35.364ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
226242025-11-12T20:56:35.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226252025-11-12T20:56:35.367ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
226262025-11-12T20:56:35.371ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = SledList step id = Fetch
226272025-11-12T20:56:35.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
226282025-11-12T20:56:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
226292025-11-12T20:56:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
226302025-11-12T20:56:35.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226312025-11-12T20:56:35.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226322025-11-12T20:56:35.373ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
226332025-11-12T20:56:35.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
226342025-11-12T20:56:35.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226352025-11-12T20:56:35.377ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
226362025-11-12T20:56:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
226372025-11-12T20:56:35.378ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
226382025-11-12T20:56:35.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
226392025-11-12T20:56:35.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
226402025-11-12T20:56:35.384ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
226412025-11-12T20:56:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
226422025-11-12T20:56:35.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226432025-11-12T20:56:35.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226442025-11-12T20:56:35.385ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
226452025-11-12T20:56:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226462025-11-12T20:56:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
226472025-11-12T20:56:35.386ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
226482025-11-12T20:56:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
226492025-11-12T20:56:35.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226502025-11-12T20:56:35.389ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 111.67423ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
226512025-11-12T20:56:35.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
226522025-11-12T20:56:35.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
226532025-11-12T20:56:35.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
226542025-11-12T20:56:35.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
226552025-11-12T20:56:35.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226562025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226572025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
226582025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
226592025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
226602025-11-12T20:56:35.394ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
226612025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
226622025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
226632025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
226642025-11-12T20:56:35.394ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = DeployNexusRecords step id = Ensure
226652025-11-12T20:56:35.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
226662025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
226672025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
226682025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
226692025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
226702025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
226712025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
226722025-11-12T20:56:35.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
226732025-11-12T20:56:35.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226742025-11-12T20:56:35.397ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226752025-11-12T20:56:35.397ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226762025-11-12T20:56:35.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226772025-11-12T20:56:35.399ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
226782025-11-12T20:56:35.399ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
226792025-11-12T20:56:35.399ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
226802025-11-12T20:56:35.399ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 122.021408ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
226812025-11-12T20:56:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
226822025-11-12T20:56:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
226832025-11-12T20:56:35.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
226842025-11-12T20:56:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
226852025-11-12T20:56:35.401ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
226862025-11-12T20:56:35.401ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226872025-11-12T20:56:35.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
226882025-11-12T20:56:35.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
226892025-11-12T20:56:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
226902025-11-12T20:56:35.403ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226912025-11-12T20:56:35.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
226922025-11-12T20:56:35.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226932025-11-12T20:56:35.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226942025-11-12T20:56:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226952025-11-12T20:56:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
226962025-11-12T20:56:35.405ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
226972025-11-12T20:56:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
226982025-11-12T20:56:35.405ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
226992025-11-12T20:56:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
227002025-11-12T20:56:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
227012025-11-12T20:56:35.407ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
227022025-11-12T20:56:35.407ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
227032025-11-12T20:56:35.407ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
227042025-11-12T20:56:35.407ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 130.307832ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
227052025-11-12T20:56:35.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
227062025-11-12T20:56:35.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227072025-11-12T20:56:35.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227082025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
227092025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
227102025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
227112025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
227122025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
227132025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 134.804988ms iteration = 12 status = {"generation":2}
227142025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
227152025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
227162025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
227172025-11-12T20:56:35.409ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
227182025-11-12T20:56:35.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
227192025-11-12T20:56:35.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227202025-11-12T20:56:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
227212025-11-12T20:56:35.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227222025-11-12T20:56:35.414ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227232025-11-12T20:56:35.417ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
227242025-11-12T20:56:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
227252025-11-12T20:56:35.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
227262025-11-12T20:56:35.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227272025-11-12T20:56:35.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227282025-11-12T20:56:35.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
227292025-11-12T20:56:35.419ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
227302025-11-12T20:56:35.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
227312025-11-12T20:56:35.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
227322025-11-12T20:56:35.421ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
227332025-11-12T20:56:35.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
227342025-11-12T20:56:35.424ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227352025-11-12T20:56:35.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227362025-11-12T20:56:35.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227372025-11-12T20:56:35.428ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
227382025-11-12T20:56:35.428ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
227392025-11-12T20:56:35.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
227402025-11-12T20:56:35.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227412025-11-12T20:56:35.433ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
227422025-11-12T20:56:35.433ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 155.982306ms iteration = 34 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
227432025-11-12T20:56:35.435ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 157.609649ms iteration = 37 status = {"physical_disks_added":0}
227442025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
227452025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
227462025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
227472025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 163.862431ms iteration = 12 status = {"generation":1}
227482025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
227492025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227502025-11-12T20:56:35.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227512025-11-12T20:56:35.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227522025-11-12T20:56:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227532025-11-12T20:56:35.449ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
227542025-11-12T20:56:35.449ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
227552025-11-12T20:56:35.449ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 172.132735ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
227562025-11-12T20:56:35.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227572025-11-12T20:56:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
227582025-11-12T20:56:35.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227592025-11-12T20:56:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227602025-11-12T20:56:35.474ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 196.758592ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
227612025-11-12T20:56:35.474ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 196.966227ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
227622025-11-12T20:56:35.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227632025-11-12T20:56:35.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227642025-11-12T20:56:35.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227652025-11-12T20:56:35.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227662025-11-12T20:56:35.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227672025-11-12T20:56:35.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227682025-11-12T20:56:35.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227692025-11-12T20:56:35.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227702025-11-12T20:56:35.490ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
227712025-11-12T20:56:35.490ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
227722025-11-12T20:56:35.490ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
227732025-11-12T20:56:35.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227742025-11-12T20:56:35.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
227752025-11-12T20:56:35.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227762025-11-12T20:56:35.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227772025-11-12T20:56:35.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227782025-11-12T20:56:35.499ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
227792025-11-12T20:56:35.499ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 224.736556ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
227802025-11-12T20:56:35.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227812025-11-12T20:56:35.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227822025-11-12T20:56:35.501ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = SledAgent step id = Ensure
227832025-11-12T20:56:35.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227842025-11-12T20:56:35.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227852025-11-12T20:56:35.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227862025-11-12T20:56:35.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227872025-11-12T20:56:35.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227882025-11-12T20:56:35.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227892025-11-12T20:56:35.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227902025-11-12T20:56:35.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227912025-11-12T20:56:35.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227922025-11-12T20:56:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227932025-11-12T20:56:35.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227942025-11-12T20:56:35.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227952025-11-12T20:56:35.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227962025-11-12T20:56:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227972025-11-12T20:56:35.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227982025-11-12T20:56:35.545ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
227992025-11-12T20:56:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
228002025-11-12T20:56:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
228012025-11-12T20:56:35.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228022025-11-12T20:56:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228032025-11-12T20:56:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228042025-11-12T20:56:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228052025-11-12T20:56:35.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228062025-11-12T20:56:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
228072025-11-12T20:56:35.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228082025-11-12T20:56:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228092025-11-12T20:56:35.561ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
228102025-11-12T20:56:35.561ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
228112025-11-12T20:56:35.561ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 283.588689ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
228122025-11-12T20:56:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228132025-11-12T20:56:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228142025-11-12T20:56:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228152025-11-12T20:56:35.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
228162025-11-12T20:56:35.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228172025-11-12T20:56:35.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228182025-11-12T20:56:35.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228192025-11-12T20:56:35.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228202025-11-12T20:56:35.574ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
228212025-11-12T20:56:35.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228222025-11-12T20:56:35.579ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
228232025-11-12T20:56:35.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228242025-11-12T20:56:35.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228252025-11-12T20:56:35.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228262025-11-12T20:56:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228272025-11-12T20:56:35.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228282025-11-12T20:56:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
228292025-11-12T20:56:35.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228302025-11-12T20:56:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228312025-11-12T20:56:35.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228322025-11-12T20:56:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228332025-11-12T20:56:35.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228342025-11-12T20:56:35.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
228352025-11-12T20:56:35.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228362025-11-12T20:56:35.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228372025-11-12T20:56:35.597ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
228382025-11-12T20:56:35.597ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
228392025-11-12T20:56:35.597ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a7d0d5ad-61aa-46be-b0c9-2976d6aa503b parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
228402025-11-12T20:56:35.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228412025-11-12T20:56:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228422025-11-12T20:56:35.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228432025-11-12T20:56:35.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
228442025-11-12T20:56:35.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
228452025-11-12T20:56:35.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
228462025-11-12T20:56:35.606ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
228472025-11-12T20:56:35.606ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
228482025-11-12T20:56:35.606ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 328.964394ms iteration = 28 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
228492025-11-12T20:56:35.638ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
228502025-11-12T20:56:35.638ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
228512025-11-12T20:56:35.639ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fbe5218-324c-40b1-bafd-d57005515375", "content-length": "26", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228522025-11-12T20:56:35.639ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
228532025-11-12T20:56:35.640ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
228542025-11-12T20:56:35.649ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
228552025-11-12T20:56:35.651ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
228562025-11-12T20:56:35.651ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
228572025-11-12T20:56:35.651ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "61140e21-8906-4069-bd3f-782f9ae34aef", "content-length": "372", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228582025-11-12T20:56:35.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
228592025-11-12T20:56:35.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
228602025-11-12T20:56:35.666ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
228612025-11-12T20:56:35.666ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d2b5797-d0cf-44c8-a29a-e1d717e95cf2", "content-length": "78", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228622025-11-12T20:56:35.666ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
228632025-11-12T20:56:35.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
228642025-11-12T20:56:35.667ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
228652025-11-12T20:56:35.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
228662025-11-12T20:56:35.669ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
228672025-11-12T20:56:35.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
228682025-11-12T20:56:35.671ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
228692025-11-12T20:56:35.672ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
228702025-11-12T20:56:35.672ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
228712025-11-12T20:56:35.673ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
228722025-11-12T20:56:35.673ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
228732025-11-12T20:56:35.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
228742025-11-12T20:56:35.676ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
228752025-11-12T20:56:35.676ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
228762025-11-12T20:56:35.676ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
228772025-11-12T20:56:35.676ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
228782025-11-12T20:56:35.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
228792025-11-12T20:56:35.676ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "daf76ea9-86ca-4d44-a9c8-7afb1cafea38", "content-length": "81", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228802025-11-12T20:56:35.677ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
228812025-11-12T20:56:35.677ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
228822025-11-12T20:56:35.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
228832025-11-12T20:56:35.686ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
228842025-11-12T20:56:35.687ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "53c8b3c0-b62e-4c40-a0c7-a6a21a211330", "content-length": "691", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228852025-11-12T20:56:35.687ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
228862025-11-12T20:56:35.687ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
228872025-11-12T20:56:35.687ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
228882025-11-12T20:56:35.689ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 411.554151ms iteration = 23 status = {}
228892025-11-12T20:56:35.703ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
228902025-11-12T20:56:35.703ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
228912025-11-12T20:56:35.704ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd172e9d-7d02-416b-ac34-0fe44fd1bb9c", "content-length": "26", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228922025-11-12T20:56:35.704ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
228932025-11-12T20:56:35.706ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
228942025-11-12T20:56:35.706ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
228952025-11-12T20:56:35.706ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "6ac5d060-81e2-4705-b5c8-9be0da8ae825", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
228962025-11-12T20:56:35.706ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
228972025-11-12T20:56:35.708ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
228982025-11-12T20:56:35.708ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 419.124946ms iteration = 12 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
228992025-11-12T20:56:35.709ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
229002025-11-12T20:56:35.709ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 431.60559ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
229012025-11-12T20:56:35.717ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
229022025-11-12T20:56:35.717ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
229032025-11-12T20:56:35.717ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e04c79da-1b3b-4352-8065-ca6669bb2efd", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
229042025-11-12T20:56:35.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229052025-11-12T20:56:35.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229062025-11-12T20:56:35.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229072025-11-12T20:56:35.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229082025-11-12T20:56:35.723ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229092025-11-12T20:56:35.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229102025-11-12T20:56:35.725ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
229112025-11-12T20:56:35.725ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
229122025-11-12T20:56:35.725ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a511551-65c4-42d8-9560-669e34b1e246", "content-length": "81", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
229132025-11-12T20:56:35.727ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
229142025-11-12T20:56:35.727ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
229152025-11-12T20:56:35.727ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d28648fe-8ef1-4855-b3ac-5a0271ca0d40", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
229162025-11-12T20:56:35.730ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 452.308145ms iteration = 34 status = {}
229172025-11-12T20:56:35.730ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Dependency
229182025-11-12T20:56:35.730ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
229192025-11-12T20:56:35.730ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229202025-11-12T20:56:35.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229212025-11-12T20:56:35.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229222025-11-12T20:56:35.734ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
229232025-11-12T20:56:35.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229242025-11-12T20:56:35.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229252025-11-12T20:56:35.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229262025-11-12T20:56:35.736ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
229272025-11-12T20:56:35.736ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
229282025-11-12T20:56:35.736ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bd33a3b-4596-428a-971b-350040efda2c", "content-length": "81", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
229292025-11-12T20:56:35.738ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 461.37232ms iteration = 23 status = {"disabled":false,"errors":[],"sps":[]}
229302025-11-12T20:56:35.739ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
229312025-11-12T20:56:35.739ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229322025-11-12T20:56:35.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229332025-11-12T20:56:35.809ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
229342025-11-12T20:56:35.813ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
229352025-11-12T20:56:35.815ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
229362025-11-12T20:56:35.815ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 541.601427ms iteration = 23 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
229372025-11-12T20:56:35.816ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "482140c5-1cfd-4f44-a93f-72b5998d413f", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
229382025-11-12T20:56:35.818ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = FirewallRules step id = Ensure
229392025-11-12T20:56:35.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229402025-11-12T20:56:35.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229412025-11-12T20:56:35.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229422025-11-12T20:56:35.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229432025-11-12T20:56:35.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229442025-11-12T20:56:35.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229452025-11-12T20:56:35.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229462025-11-12T20:56:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229472025-11-12T20:56:35.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229482025-11-12T20:56:35.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229492025-11-12T20:56:35.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229502025-11-12T20:56:35.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229512025-11-12T20:56:35.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229522025-11-12T20:56:35.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
229532025-11-12T20:56:35.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229542025-11-12T20:56:35.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229552025-11-12T20:56:35.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229562025-11-12T20:56:35.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229572025-11-12T20:56:35.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229582025-11-12T20:56:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229592025-11-12T20:56:35.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229602025-11-12T20:56:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
229612025-11-12T20:56:35.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229622025-11-12T20:56:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229632025-11-12T20:56:35.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229642025-11-12T20:56:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229652025-11-12T20:56:35.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229662025-11-12T20:56:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229672025-11-12T20:56:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229682025-11-12T20:56:35.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229692025-11-12T20:56:35.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229702025-11-12T20:56:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229712025-11-12T20:56:35.859ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1591, tv_nsec: 357215990 } })
229722025-11-12T20:56:35.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229732025-11-12T20:56:35.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229742025-11-12T20:56:35.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229752025-11-12T20:56:35.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229762025-11-12T20:56:35.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229772025-11-12T20:56:35.866ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 592.081492ms iteration = 23 status = {}
229782025-11-12T20:56:35.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
229792025-11-12T20:56:35.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229802025-11-12T20:56:35.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229812025-11-12T20:56:35.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229822025-11-12T20:56:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229832025-11-12T20:56:35.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229842025-11-12T20:56:35.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229852025-11-12T20:56:35.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229862025-11-12T20:56:35.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229872025-11-12T20:56:35.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229882025-11-12T20:56:35.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229892025-11-12T20:56:35.879ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
229902025-11-12T20:56:35.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229912025-11-12T20:56:35.880ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3fc42a5b-c2cc-461f-8f68-1d2569001412", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
229922025-11-12T20:56:35.880ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
229932025-11-12T20:56:35.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229942025-11-12T20:56:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
229952025-11-12T20:56:35.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
229962025-11-12T20:56:35.882ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c5009a69-2b68-499a-97c6-6b201b276cd4", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
229972025-11-12T20:56:35.882ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
229982025-11-12T20:56:35.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229992025-11-12T20:56:35.884ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
230002025-11-12T20:56:35.884ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 610.065288ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
230012025-11-12T20:56:35.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
230022025-11-12T20:56:35.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230032025-11-12T20:56:35.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230042025-11-12T20:56:35.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230052025-11-12T20:56:35.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230062025-11-12T20:56:35.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230072025-11-12T20:56:35.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230082025-11-12T20:56:35.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230092025-11-12T20:56:35.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
230102025-11-12T20:56:35.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230112025-11-12T20:56:35.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230122025-11-12T20:56:35.901ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
230132025-11-12T20:56:35.901ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
230142025-11-12T20:56:35.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230152025-11-12T20:56:35.902ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "99d5e3a1-b479-4e9d-8490-c74a3e282b95", "content-length": "372", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
230162025-11-12T20:56:35.902ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
230172025-11-12T20:56:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230182025-11-12T20:56:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
230192025-11-12T20:56:35.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230202025-11-12T20:56:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230212025-11-12T20:56:35.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230222025-11-12T20:56:35.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230232025-11-12T20:56:35.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230242025-11-12T20:56:35.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230252025-11-12T20:56:35.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230262025-11-12T20:56:35.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230272025-11-12T20:56:35.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230282025-11-12T20:56:35.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230292025-11-12T20:56:35.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230302025-11-12T20:56:35.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230312025-11-12T20:56:35.913ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
230322025-11-12T20:56:35.913ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "06717ca1-3210-40e7-a777-aca34b78ff59", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230332025-11-12T20:56:35.913ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230342025-11-12T20:56:35.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230352025-11-12T20:56:35.914ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
230362025-11-12T20:56:35.914ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
230372025-11-12T20:56:35.914ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "baf0eaee-a894-4376-a8d1-c291eea9c071", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
230382025-11-12T20:56:35.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
230392025-11-12T20:56:35.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
230402025-11-12T20:56:35.916ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5f1f1b6b-caf4-4de8-9338-d50236544739", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230412025-11-12T20:56:35.916ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230422025-11-12T20:56:35.916ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
230432025-11-12T20:56:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
230442025-11-12T20:56:35.918ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "77da4251-3a24-43c5-ab99-7b21aef33bcb", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230452025-11-12T20:56:35.918ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230462025-11-12T20:56:35.919ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
230472025-11-12T20:56:35.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
230482025-11-12T20:56:35.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
230492025-11-12T20:56:35.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230502025-11-12T20:56:35.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230512025-11-12T20:56:35.920ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
230522025-11-12T20:56:35.920ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 646.406639ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
230532025-11-12T20:56:35.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230542025-11-12T20:56:35.922ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
230552025-11-12T20:56:35.922ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
230562025-11-12T20:56:35.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230572025-11-12T20:56:35.922ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c2ec11c1-5910-4c71-8bd9-19314a0862e8", "date": "Wed, 12 Nov 2025 20:56:35 GMT"} })
230582025-11-12T20:56:35.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230592025-11-12T20:56:35.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230602025-11-12T20:56:35.924ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 194.860693ms iteration = 35 status = {}
230612025-11-12T20:56:35.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230622025-11-12T20:56:35.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
230632025-11-12T20:56:35.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230642025-11-12T20:56:35.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230652025-11-12T20:56:35.936ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
230662025-11-12T20:56:35.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230672025-11-12T20:56:35.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230682025-11-12T20:56:35.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230692025-11-12T20:56:35.940ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
230702025-11-12T20:56:35.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230712025-11-12T20:56:35.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230722025-11-12T20:56:35.941ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
230732025-11-12T20:56:35.941ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
230742025-11-12T20:56:35.942ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
230752025-11-12T20:56:35.958ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
230762025-11-12T20:56:35.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
230772025-11-12T20:56:35.960ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
230782025-11-12T20:56:35.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
230792025-11-12T20:56:35.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
230802025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
230812025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
230822025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
230832025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
230842025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
230852025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
230862025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
230872025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
230882025-11-12T20:56:35.971ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
230892025-11-12T20:56:35.972ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
230902025-11-12T20:56:35.972ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
230912025-11-12T20:56:35.972ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
230922025-11-12T20:56:35.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
230932025-11-12T20:56:35.974ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
230942025-11-12T20:56:35.974ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
230952025-11-12T20:56:36.003ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
230962025-11-12T20:56:36.003ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
230972025-11-12T20:56:36.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230982025-11-12T20:56:36.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230992025-11-12T20:56:36.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231002025-11-12T20:56:36.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231012025-11-12T20:56:36.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231022025-11-12T20:56:36.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
231032025-11-12T20:56:36.042ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
231042025-11-12T20:56:36.043ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "363c64ff-a5cf-478f-98f3-5aa978ab6c15", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
231052025-11-12T20:56:36.045ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
231062025-11-12T20:56:36.045ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = Dns step id = Ensure
231072025-11-12T20:56:36.045ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
231082025-11-12T20:56:36.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231092025-11-12T20:56:36.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231102025-11-12T20:56:36.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231112025-11-12T20:56:36.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231122025-11-12T20:56:36.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231132025-11-12T20:56:36.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231142025-11-12T20:56:36.052ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
231152025-11-12T20:56:36.052ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
231162025-11-12T20:56:36.052ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
231172025-11-12T20:56:36.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231182025-11-12T20:56:36.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231192025-11-12T20:56:36.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231202025-11-12T20:56:36.057ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231212025-11-12T20:56:36.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231222025-11-12T20:56:36.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231232025-11-12T20:56:36.060ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
231242025-11-12T20:56:36.060ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
231252025-11-12T20:56:36.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231262025-11-12T20:56:36.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231272025-11-12T20:56:36.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231282025-11-12T20:56:36.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231292025-11-12T20:56:36.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231302025-11-12T20:56:36.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231312025-11-12T20:56:36.068ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
231322025-11-12T20:56:36.068ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
231332025-11-12T20:56:36.068ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
231342025-11-12T20:56:36.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231352025-11-12T20:56:36.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231362025-11-12T20:56:36.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231372025-11-12T20:56:36.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231382025-11-12T20:56:36.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231392025-11-12T20:56:36.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231402025-11-12T20:56:36.074ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
231412025-11-12T20:56:36.074ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
231422025-11-12T20:56:36.074ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
231432025-11-12T20:56:36.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231442025-11-12T20:56:36.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231452025-11-12T20:56:36.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231462025-11-12T20:56:36.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231472025-11-12T20:56:36.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231482025-11-12T20:56:36.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231492025-11-12T20:56:36.086ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
231502025-11-12T20:56:36.086ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
231512025-11-12T20:56:36.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231522025-11-12T20:56:36.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231532025-11-12T20:56:36.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231542025-11-12T20:56:36.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231552025-11-12T20:56:36.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231562025-11-12T20:56:36.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231572025-11-12T20:56:36.094ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
231582025-11-12T20:56:36.094ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
231592025-11-12T20:56:36.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231602025-11-12T20:56:36.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231612025-11-12T20:56:36.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231622025-11-12T20:56:36.098ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
231632025-11-12T20:56:36.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231642025-11-12T20:56:36.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231652025-11-12T20:56:36.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231662025-11-12T20:56:36.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231672025-11-12T20:56:36.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231682025-11-12T20:56:36.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231692025-11-12T20:56:36.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231702025-11-12T20:56:36.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
231712025-11-12T20:56:36.108ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
231722025-11-12T20:56:36.108ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
231732025-11-12T20:56:36.108ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = OmicronZones step id = Cleanup
231742025-11-12T20:56:36.108ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = OmicronZones step id = Cleanup
231752025-11-12T20:56:36.108ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = PhysicalDisks step id = Cleanup
231762025-11-12T20:56:36.108ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = Clickhouse step id = Ensure
231772025-11-12T20:56:36.108ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = Clickhouse step id = Ensure
231782025-11-12T20:56:36.140ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
231792025-11-12T20:56:36.147ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "633cf7aa-9660-430f-b49f-3ce87805f97a", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
231802025-11-12T20:56:36.148ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = SupportBundles step id = Cleanup
231812025-11-12T20:56:36.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231822025-11-12T20:56:36.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231832025-11-12T20:56:36.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231842025-11-12T20:56:36.151ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
231852025-11-12T20:56:36.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231862025-11-12T20:56:36.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231872025-11-12T20:56:36.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231882025-11-12T20:56:36.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231892025-11-12T20:56:36.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231902025-11-12T20:56:36.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
231912025-11-12T20:56:36.196ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7c745401-caeb-4c5c-bc3f-20a2a7861f9c", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
231922025-11-12T20:56:36.196ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
231932025-11-12T20:56:36.196ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
231942025-11-12T20:56:36.198ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = OmicronZones step id = Cleanup
231952025-11-12T20:56:36.198ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
231962025-11-12T20:56:36.198ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
231972025-11-12T20:56:36.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
231982025-11-12T20:56:36.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
231992025-11-12T20:56:36.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232002025-11-12T20:56:36.202ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
232012025-11-12T20:56:36.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
232022025-11-12T20:56:36.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
232032025-11-12T20:56:36.205ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f46e0941-8350-47f7-80bf-8efa00e1906d", "content-length": "682", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232042025-11-12T20:56:36.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232052025-11-12T20:56:36.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232062025-11-12T20:56:36.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232072025-11-12T20:56:36.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232082025-11-12T20:56:36.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232092025-11-12T20:56:36.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232102025-11-12T20:56:36.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232112025-11-12T20:56:36.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232122025-11-12T20:56:36.221ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 944.240545ms iteration = 23 status = {}
232132025-11-12T20:56:36.238ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
232142025-11-12T20:56:36.238ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
232152025-11-12T20:56:36.238ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = Cockroach step id = Ensure
232162025-11-12T20:56:36.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
232172025-11-12T20:56:36.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
232182025-11-12T20:56:36.242ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
232192025-11-12T20:56:36.242ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
232202025-11-12T20:56:36.256ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
232212025-11-12T20:56:36.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e2bf6583-ec1f-4eea-b3b9-089ca9174395 step component = MgsUpdates step id = Ensure
232222025-11-12T20:56:36.256ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 35 reason = Dependency
232232025-11-12T20:56:36.256ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Dependency
232242025-11-12T20:56:36.256ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
232252025-11-12T20:56:36.256ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
232262025-11-12T20:56:36.256ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 982.297341ms iteration = 13 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2780,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":97062342,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":97062292,"secs":0}},"event_index":1,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":97089803,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":23210600,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":23210560,"secs":0}},"event_index":2,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":120320444,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":106458835,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":106458785,"secs":0}},"event_index":3,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":226801000,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":317277061,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":317277021,"secs":0}},"event_index":4,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":544094271,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":226752468,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":226752428,"secs":0}},"event_index":5,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":770857329,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":63296038,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":63295998,"secs":0}},"event_index":6,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":834160527,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5510,"secs":0}},"event_index":7,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":834177488,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2520,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2480,"secs":0}},"event_index":8,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":834182818,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3510,"secs":0}},"event_index":9,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":834188408,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1890,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1850,"secs":0}},"event_index":10,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":834192348,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39527233,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":39527203,"secs":0}},"event_index":11,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":873724911,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49890951,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":49890911,"secs":0}},"event_index":12,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":923627923,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40618191,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":40618151,"secs":0}},"event_index":13,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":964260204,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":17670357,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":17670307,"secs":0}},"event_index":14,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":981940482,"secs":0}},{"data":{"attempt_elapsed":{"nanos":4940,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4890,"secs":0}},"event_index":15,"execution_id":"08d1de44-1718-4835-b75d-89c69eac9614","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":981954812,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
232272025-11-12T20:56:36.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
232282025-11-12T20:56:36.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
232292025-11-12T20:56:36.258ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232302025-11-12T20:56:36.258ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232312025-11-12T20:56:36.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
232322025-11-12T20:56:36.260ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
232332025-11-12T20:56:36.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
232342025-11-12T20:56:36.261ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
232352025-11-12T20:56:36.261ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
232362025-11-12T20:56:36.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
232372025-11-12T20:56:36.262ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
232382025-11-12T20:56:36.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
232392025-11-12T20:56:36.264ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232402025-11-12T20:56:36.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
232412025-11-12T20:56:36.266ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
232422025-11-12T20:56:36.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
232432025-11-12T20:56:36.275ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
232442025-11-12T20:56:36.275ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:50687
232452025-11-12T20:56:36.275ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50687 req_id = dceaf849-7a3e-4b12-af6c-5ca10e651143 uri = /metrics/producers version = 1.0.0
232462025-11-12T20:56:36.275ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50687 req_id = dceaf849-7a3e-4b12-af6c-5ca10e651143 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
232472025-11-12T20:56:36.275ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50687 req_id = dceaf849-7a3e-4b12-af6c-5ca10e651143 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
232482025-11-12T20:56:36.291ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0, 0), (fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51, 0), (d51aac2a-2b83-467c-9b97-0e793473a9af, 0), (7a3a620a-d640-4ac9-9394-c0f9ac495599, 0)] collection_id = 8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0
232492025-11-12T20:56:36.291ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
232502025-11-12T20:56:36.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
232512025-11-12T20:56:36.306ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
232522025-11-12T20:56:36.306ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 50.405814ms iteration = 35 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
232532025-11-12T20:56:36.334ZINFOnexus: assigned collector to new producer collector_id = ecb0f932-e2e8-41d3-85a9-494e5cae5bce file = nexus/src/app/oximeter.rs:76 producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
232542025-11-12T20:56:36.334ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 59020 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:50687 req_id = dceaf849-7a3e-4b12-af6c-5ca10e651143 response_code = 201 uri = /metrics/producers
232552025-11-12T20:56:36.334ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "dceaf849-7a3e-4b12-af6c-5ca10e651143", "content-length": "41", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232562025-11-12T20:56:36.336ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
232572025-11-12T20:56:36.422ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 8d1c8e0b-1c76-4ab7-91f3-1c37c7dfbfc0 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
232582025-11-12T20:56:36.422ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
232592025-11-12T20:56:36.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
232602025-11-12T20:56:36.426ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51, 0), (d51aac2a-2b83-467c-9b97-0e793473a9af, 0), (7a3a620a-d640-4ac9-9394-c0f9ac495599, 0), (229ed602-3638-48e4-9414-366c12eb36a9, 0)] collection_id = fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51
232612025-11-12T20:56:36.426ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
232622025-11-12T20:56:36.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
232632025-11-12T20:56:36.540ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = fcb4d9bf-6a72-44b5-a484-bb5b0a1aba51 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
232642025-11-12T20:56:36.540ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
232652025-11-12T20:56:36.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
232662025-11-12T20:56:36.544ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(d51aac2a-2b83-467c-9b97-0e793473a9af, 0), (7a3a620a-d640-4ac9-9394-c0f9ac495599, 0), (229ed602-3638-48e4-9414-366c12eb36a9, 0)]
232672025-11-12T20:56:36.544ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 41918049 } })
232682025-11-12T20:56:36.576ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 73502757 } })
232692025-11-12T20:56:36.576ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
232702025-11-12T20:56:36.576ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
232712025-11-12T20:56:36.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
232722025-11-12T20:56:36.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6717bcff-48c9-4324-a1f8-ddefaf194046", "content-length": "476", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232732025-11-12T20:56:36.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
232742025-11-12T20:56:36.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0a4c174-a111-4f56-b37d-c2677b0d9d65", "content-length": "734", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232752025-11-12T20:56:36.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
232762025-11-12T20:56:36.578ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b00b531-49fe-4784-8233-91237e409b22", "content-length": "111", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232772025-11-12T20:56:36.578ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
232782025-11-12T20:56:36.578ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0437664b-3d85-491b-bba9-c7031ad58509", "content-length": "111", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232792025-11-12T20:56:36.578ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
232802025-11-12T20:56:36.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "356f0fe8-bb16-4dd3-a60b-fe57d4441352", "content-length": "163", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232812025-11-12T20:56:36.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
232822025-11-12T20:56:36.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ed63eea-6ecd-4224-a888-c75dfe07e231", "content-length": "163", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232832025-11-12T20:56:36.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
232842025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ff014aa-31a4-4a86-8f7a-c84b0cdd9b8a", "content-length": "166", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232852025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
232862025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d645e737-e3e2-44ee-93c7-7d85ee75713b", "content-length": "166", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232872025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
232882025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e7aedd3-eed0-44b7-99b5-10648668eba3", "content-length": "702", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232892025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
232902025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "76edd9ed-5fa8-4e30-b5d9-3fd3d204476b", "content-length": "727", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232912025-11-12T20:56:36.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
232922025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "906ff6cf-fae6-4380-acf4-d725af333bf5", "content-length": "729", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232932025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
232942025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56d9f768-8974-4ce4-bfd2-9dbed407b620", "content-length": "728", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232952025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
232962025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf997f9-ab09-4719-957a-d0a1c81ebeef", "content-length": "722", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232972025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
232982025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "e71066aa-fb36-499f-ac17-16cbd85d4df2", "content-length": "10", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
232992025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
233002025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b152d6fb-e1b5-4f55-acdb-729803f130aa", "content-length": "144", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233012025-11-12T20:56:36.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
233022025-11-12T20:56:36.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "585e5abc-ae2b-4f0a-94c3-970b48f5ec3e", "content-length": "147", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233032025-11-12T20:56:36.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
233042025-11-12T20:56:36.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "efcefd2e-07a3-4ef2-af32-4859a33ceb4f", "content-length": "109", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233052025-11-12T20:56:36.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
233062025-11-12T20:56:36.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e71ce57-22dc-4189-bafb-481f1e6cd0f7", "content-length": "109", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233072025-11-12T20:56:36.583ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
233082025-11-12T20:56:36.583ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e31dcb7-d252-4dda-9fbb-558d70606e3c", "content-length": "163", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233092025-11-12T20:56:36.583ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
233102025-11-12T20:56:36.583ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da81ecd4-617b-4505-8533-547553ebf9a9", "content-length": "163", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233112025-11-12T20:56:36.583ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
233122025-11-12T20:56:36.584ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c2e16c7-8eb7-48ab-954d-b001dcd284d7", "content-length": "166", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233132025-11-12T20:56:36.584ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
233142025-11-12T20:56:36.584ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bef88bc-183c-40a5-8ce4-3948029722ac", "content-length": "166", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233152025-11-12T20:56:36.584ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
233162025-11-12T20:56:36.584ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f99c43a2-c8ff-413c-8e00-773283a4d514", "content-length": "702", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233172025-11-12T20:56:36.584ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
233182025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3c254cd-b75f-4b55-bc74-bb5ead5d08d5", "content-length": "727", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233192025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
233202025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d931f4dc-a711-408f-bc4b-c105dcabfba2", "content-length": "729", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233212025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
233222025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cd2c184-660a-4b84-93cc-1e00afb8043b", "content-length": "728", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233232025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
233242025-11-12T20:56:36.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
233252025-11-12T20:56:36.587ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233262025-11-12T20:56:36.589ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
233272025-11-12T20:56:36.589ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
233282025-11-12T20:56:36.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
233292025-11-12T20:56:36.591ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233302025-11-12T20:56:36.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
233312025-11-12T20:56:36.593ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
233322025-11-12T20:56:36.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
233332025-11-12T20:56:36.628ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
233342025-11-12T20:56:36.628ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
233352025-11-12T20:56:36.726ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c077f46-3ba3-4a98-a42a-f1cecd325b14", "content-length": "95176", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233362025-11-12T20:56:36.728ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
233372025-11-12T20:56:36.728ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
233382025-11-12T20:56:36.728ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
233392025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
233402025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
233412025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
233422025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
233432025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
233442025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "29f26fa9-2af1-446e-bf3b-3f765f591b64", "content-length": "64", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233452025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca0dc35b-86e1-4374-afc3-b01b583a281d", "content-length": "64", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233462025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "15fb09f5-e827-4aed-b736-4b648da9e87d", "content-length": "64", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233472025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
233482025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ebfb910-4b18-43c0-8bb6-b6936d0154a1", "content-length": "64", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233492025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
233502025-11-12T20:56:36.728ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
233512025-11-12T20:56:36.729ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
233522025-11-12T20:56:36.729ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e1b89a0-3887-4310-bd51-d4ae34e1c2fd", "content-length": "64", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233532025-11-12T20:56:36.729ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
233542025-11-12T20:56:36.770ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0072124-f6ab-4cc5-8bf4-2e866c81282c", "content-length": "407883", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233552025-11-12T20:56:36.771ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2e42c8b-78f9-4635-91db-07b9d8561df1", "content-length": "411678", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233562025-11-12T20:56:36.772ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1396bb8b-c474-4016-a90d-a506f564d949", "content-length": "436513", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233572025-11-12T20:56:36.774ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c5d3fec-9a9c-4265-b6b2-a2002aed2c92", "content-length": "413251", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233582025-11-12T20:56:36.777ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0b8547f-f9d4-4dcc-9a2a-8919bced8b66", "content-length": "417295", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233592025-11-12T20:56:36.779ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
233602025-11-12T20:56:36.779ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
233612025-11-12T20:56:36.779ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
233622025-11-12T20:56:36.779ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
233632025-11-12T20:56:36.780ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
233642025-11-12T20:56:36.810ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
233652025-11-12T20:56:36.810ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
233662025-11-12T20:56:36.813ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d33c513-ae0f-4cf2-a00d-1297666cff4e", "content-length": "128", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233672025-11-12T20:56:36.815ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
233682025-11-12T20:56:36.907ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
233692025-11-12T20:56:36.907ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
233702025-11-12T20:56:36.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ea484fd-c3be-43e5-8420-7780fcad2ee5", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233712025-11-12T20:56:36.908ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
233722025-11-12T20:56:36.910ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
233732025-11-12T20:56:36.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
233742025-11-12T20:56:36.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef742c08-9466-4d14-b924-7db3648c0cb1", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233752025-11-12T20:56:36.911ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
233762025-11-12T20:56:36.913ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
233772025-11-12T20:56:36.913ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
233782025-11-12T20:56:36.913ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8abb67d2-f4e1-451e-9920-3b79c77e13d7", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:56:36 GMT"} })
233792025-11-12T20:56:36.913ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
233802025-11-12T20:56:36.914ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
233812025-11-12T20:56:36.914ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
233822025-11-12T20:56:36.916ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
233832025-11-12T20:56:36.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
233842025-11-12T20:56:36.918ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233852025-11-12T20:56:36.920ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
233862025-11-12T20:56:36.920ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
233872025-11-12T20:56:36.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
233882025-11-12T20:56:37.111ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 737d712c-07bf-4cbc-9c46-02de218b0f65 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
233892025-11-12T20:56:37.111ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 737d712c-07bf-4cbc-9c46-02de218b0f65 time_started = 2025-11-12 20:56:36.576802 UTC
233902025-11-12T20:56:37.111ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 59 reason = Dependency
233912025-11-12T20:56:37.111ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
233922025-11-12T20:56:37.111ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 855.18467ms iteration = 14 status = {"collection_id":"737d712c-07bf-4cbc-9c46-02de218b0f65","time_done":"2025-11-12 20:56:36.914638 UTC","time_started":"2025-11-12 20:56:36.576802 UTC"}
233932025-11-12T20:56:37.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
233942025-11-12T20:56:37.114ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233952025-11-12T20:56:37.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
233962025-11-12T20:56:37.117ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
233972025-11-12T20:56:37.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
233982025-11-12T20:56:37.194ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
233992025-11-12T20:56:37.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
234002025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 29 reason = Dependency
234012025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 135.144437ms iteration = 59 status = {"Loaded":{"collection_id":"98e4220e-c01e-4ccf-adc3-06f59cdfaa04","time_loaded":"2025-11-12T20:56:37.111713787Z","time_started":"2025-11-12T20:56:36.679787Z"}}
234022025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 38 reason = Dependency
234032025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 18 reason = Dependency
234042025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234052025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234062025-11-12T20:56:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234072025-11-12T20:56:37.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234082025-11-12T20:56:37.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234092025-11-12T20:56:37.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234102025-11-12T20:56:37.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234112025-11-12T20:56:37.248ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234122025-11-12T20:56:37.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234132025-11-12T20:56:37.251ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
234142025-11-12T20:56:37.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234152025-11-12T20:56:37.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234162025-11-12T20:56:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
234172025-11-12T20:56:37.252ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234182025-11-12T20:56:37.252ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
234192025-11-12T20:56:37.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234202025-11-12T20:56:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234212025-11-12T20:56:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234222025-11-12T20:56:37.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234232025-11-12T20:56:37.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234242025-11-12T20:56:37.254ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234252025-11-12T20:56:37.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234262025-11-12T20:56:37.256ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234272025-11-12T20:56:37.256ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
234282025-11-12T20:56:37.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234292025-11-12T20:56:37.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234302025-11-12T20:56:37.256ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
234312025-11-12T20:56:37.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234322025-11-12T20:56:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234332025-11-12T20:56:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
234342025-11-12T20:56:37.258ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234352025-11-12T20:56:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234362025-11-12T20:56:37.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234372025-11-12T20:56:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234382025-11-12T20:56:37.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234392025-11-12T20:56:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234402025-11-12T20:56:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234412025-11-12T20:56:37.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234422025-11-12T20:56:37.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
234432025-11-12T20:56:37.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234442025-11-12T20:56:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234452025-11-12T20:56:37.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
234462025-11-12T20:56:37.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234472025-11-12T20:56:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234482025-11-12T20:56:37.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234492025-11-12T20:56:37.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234502025-11-12T20:56:37.266ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.928076ms iteration = 38 status = {"physical_disks_added":0}
234512025-11-12T20:56:37.267ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
234522025-11-12T20:56:37.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234532025-11-12T20:56:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234542025-11-12T20:56:37.268ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
234552025-11-12T20:56:37.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234562025-11-12T20:56:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234572025-11-12T20:56:37.269ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
234582025-11-12T20:56:37.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234592025-11-12T20:56:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234602025-11-12T20:56:37.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234612025-11-12T20:56:37.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234622025-11-12T20:56:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234632025-11-12T20:56:37.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
234642025-11-12T20:56:37.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234652025-11-12T20:56:37.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234662025-11-12T20:56:37.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234672025-11-12T20:56:37.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234682025-11-12T20:56:37.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234692025-11-12T20:56:37.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234702025-11-12T20:56:37.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
234712025-11-12T20:56:37.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
234722025-11-12T20:56:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
234732025-11-12T20:56:37.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
234742025-11-12T20:56:37.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234752025-11-12T20:56:37.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234762025-11-12T20:56:37.280ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
234772025-11-12T20:56:37.280ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.878127ms iteration = 18 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"98e4220e-c01e-4ccf-adc3-06f59cdfaa04","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
234782025-11-12T20:56:37.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234792025-11-12T20:56:37.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234802025-11-12T20:56:37.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234812025-11-12T20:56:37.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
234822025-11-12T20:56:37.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234832025-11-12T20:56:37.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234842025-11-12T20:56:37.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234852025-11-12T20:56:37.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
234862025-11-12T20:56:37.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234872025-11-12T20:56:37.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234882025-11-12T20:56:37.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234892025-11-12T20:56:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234902025-11-12T20:56:37.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234912025-11-12T20:56:37.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
234922025-11-12T20:56:37.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234932025-11-12T20:56:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234942025-11-12T20:56:37.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234952025-11-12T20:56:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234962025-11-12T20:56:37.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234972025-11-12T20:56:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
234982025-11-12T20:56:37.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234992025-11-12T20:56:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235002025-11-12T20:56:37.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235012025-11-12T20:56:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235022025-11-12T20:56:37.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235032025-11-12T20:56:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
235042025-11-12T20:56:37.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235052025-11-12T20:56:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235062025-11-12T20:56:37.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235072025-11-12T20:56:37.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235082025-11-12T20:56:37.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235092025-11-12T20:56:37.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235102025-11-12T20:56:37.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235112025-11-12T20:56:37.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235122025-11-12T20:56:37.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235132025-11-12T20:56:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235142025-11-12T20:56:37.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235152025-11-12T20:56:37.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235162025-11-12T20:56:37.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235172025-11-12T20:56:37.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235182025-11-12T20:56:37.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235192025-11-12T20:56:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235202025-11-12T20:56:37.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235212025-11-12T20:56:37.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235222025-11-12T20:56:37.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235232025-11-12T20:56:37.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235242025-11-12T20:56:37.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235252025-11-12T20:56:37.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235262025-11-12T20:56:37.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235272025-11-12T20:56:37.348ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235282025-11-12T20:56:37.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235292025-11-12T20:56:37.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235302025-11-12T20:56:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235312025-11-12T20:56:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235322025-11-12T20:56:37.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235332025-11-12T20:56:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235342025-11-12T20:56:37.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235352025-11-12T20:56:37.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
235362025-11-12T20:56:37.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235372025-11-12T20:56:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235382025-11-12T20:56:37.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235392025-11-12T20:56:37.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235402025-11-12T20:56:37.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235412025-11-12T20:56:37.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
235422025-11-12T20:56:37.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235432025-11-12T20:56:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235442025-11-12T20:56:37.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235452025-11-12T20:56:37.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235462025-11-12T20:56:37.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235472025-11-12T20:56:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
235482025-11-12T20:56:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235492025-11-12T20:56:37.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235502025-11-12T20:56:37.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235512025-11-12T20:56:37.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235522025-11-12T20:56:37.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235532025-11-12T20:56:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
235542025-11-12T20:56:37.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235552025-11-12T20:56:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235562025-11-12T20:56:37.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235572025-11-12T20:56:37.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235582025-11-12T20:56:37.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235592025-11-12T20:56:37.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
235602025-11-12T20:56:37.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235612025-11-12T20:56:37.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235622025-11-12T20:56:37.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235632025-11-12T20:56:37.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235642025-11-12T20:56:37.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235652025-11-12T20:56:37.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
235662025-11-12T20:56:37.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235672025-11-12T20:56:37.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235682025-11-12T20:56:37.428ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
235692025-11-12T20:56:37.428ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
235702025-11-12T20:56:37.428ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 5dccd1be-ce4c-4b7f-af2f-d9f34c8576c0 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
235712025-11-12T20:56:37.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235722025-11-12T20:56:37.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235732025-11-12T20:56:37.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235742025-11-12T20:56:37.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
235752025-11-12T20:56:37.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235762025-11-12T20:56:37.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235772025-11-12T20:56:37.436ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
235782025-11-12T20:56:37.436ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
235792025-11-12T20:56:37.436ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 190.079829ms iteration = 29 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
235802025-11-12T20:56:37.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 311 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = d5c81bf5-91b0-47be-8901-dafaa8b545e8 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
235812025-11-12T20:56:40.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 134 reason = Timeout
235822025-11-12T20:56:40.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
235832025-11-12T20:56:40.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
235842025-11-12T20:56:40.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235852025-11-12T20:56:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
235862025-11-12T20:56:40.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
235872025-11-12T20:56:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
235882025-11-12T20:56:40.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
235892025-11-12T20:56:40.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.895038ms iteration = 134 status = {"config_updated":false}
235902025-11-12T20:56:45.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
235912025-11-12T20:56:45.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
235922025-11-12T20:56:45.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
235932025-11-12T20:56:45.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
235942025-11-12T20:56:45.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 135 reason = Timeout
235952025-11-12T20:56:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
235962025-11-12T20:56:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
235972025-11-12T20:56:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
235982025-11-12T20:56:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
235992025-11-12T20:56:45.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236002025-11-12T20:56:45.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
236012025-11-12T20:56:45.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236022025-11-12T20:56:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
236032025-11-12T20:56:45.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.707656ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
236042025-11-12T20:56:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
236052025-11-12T20:56:45.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236062025-11-12T20:56:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236072025-11-12T20:56:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
236082025-11-12T20:56:45.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
236092025-11-12T20:56:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
236102025-11-12T20:56:45.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236112025-11-12T20:56:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236122025-11-12T20:56:45.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
236132025-11-12T20:56:45.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.157931ms iteration = 135 status = {"config_updated":false}
236142025-11-12T20:56:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236152025-11-12T20:56:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
236162025-11-12T20:56:45.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236172025-11-12T20:56:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236182025-11-12T20:56:45.302ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
236192025-11-12T20:56:45.302ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.908734ms iteration = 68 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
236202025-11-12T20:56:47.913ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 6182a4a3-525a-42d9-8ae1-f9c972191a90 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
236212025-11-12T20:56:47.913ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 6182a4a3-525a-42d9-8ae1-f9c972191a90 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
236222025-11-12T20:56:47.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 6182a4a3-525a-42d9-8ae1-f9c972191a90 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
236232025-11-12T20:56:47.916ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2988 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = 6182a4a3-525a-42d9-8ae1-f9c972191a90 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
236242025-11-12T20:56:47.916ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = ded2a2de-a7f4-4b56-9717-b6c99a47090f uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
236252025-11-12T20:56:47.916ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = ded2a2de-a7f4-4b56-9717-b6c99a47090f roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
236262025-11-12T20:56:47.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = ded2a2de-a7f4-4b56-9717-b6c99a47090f resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
236272025-11-12T20:56:47.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2606 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47746 req_id = ded2a2de-a7f4-4b56-9717-b6c99a47090f response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
236282025-11-12T20:56:47.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 269 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 7e96185a-3c38-4ab5-8142-5fc7508977bb response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
236292025-11-12T20:56:50.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 60 reason = Timeout
236302025-11-12T20:56:50.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
236312025-11-12T20:56:50.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
236322025-11-12T20:56:50.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 136 reason = Timeout
236332025-11-12T20:56:50.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
236342025-11-12T20:56:50.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
236352025-11-12T20:56:50.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236362025-11-12T20:56:50.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236372025-11-12T20:56:50.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
236382025-11-12T20:56:50.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
236392025-11-12T20:56:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
236402025-11-12T20:56:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
236412025-11-12T20:56:50.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
236422025-11-12T20:56:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
236432025-11-12T20:56:50.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 46 reason = Timeout
236442025-11-12T20:56:50.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
236452025-11-12T20:56:50.283ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 98e4220e-c01e-4ccf-adc3-06f59cdfaa04 original_time_started = 2025-11-12 20:56:36.679787 UTC
236462025-11-12T20:56:50.283ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.509404ms iteration = 60 status = {"Loaded":{"collection_id":"98e4220e-c01e-4ccf-adc3-06f59cdfaa04","time_loaded":"2025-11-12T20:56:50.276112163Z","time_started":"2025-11-12T20:56:36.679787Z"}}
236472025-11-12T20:56:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
236482025-11-12T20:56:50.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
236492025-11-12T20:56:50.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.160082ms iteration = 136 status = {"config_updated":false}
236502025-11-12T20:56:50.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236512025-11-12T20:56:50.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
236522025-11-12T20:56:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
236532025-11-12T20:56:50.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
236542025-11-12T20:56:50.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.693398ms iteration = 46 status = "NoSitrep"
236552025-11-12T20:56:50.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 46 reason = Timeout
236562025-11-12T20:56:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
236572025-11-12T20:56:50.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
236582025-11-12T20:56:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
236592025-11-12T20:56:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
236602025-11-12T20:56:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236612025-11-12T20:56:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
236622025-11-12T20:56:50.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
236632025-11-12T20:56:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
236642025-11-12T20:56:50.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
236652025-11-12T20:56:50.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
236662025-11-12T20:56:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236672025-11-12T20:56:50.298ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
236682025-11-12T20:56:50.299ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
236692025-11-12T20:56:50.299ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.005588ms iteration = 46 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
236702025-11-12T20:56:55.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
236712025-11-12T20:56:55.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
236722025-11-12T20:56:55.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
236732025-11-12T20:56:55.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236742025-11-12T20:56:55.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 137 reason = Timeout
236752025-11-12T20:56:55.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
236762025-11-12T20:56:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
236772025-11-12T20:56:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236782025-11-12T20:56:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
236792025-11-12T20:56:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236802025-11-12T20:56:55.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236812025-11-12T20:56:55.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
236822025-11-12T20:56:55.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
236832025-11-12T20:56:55.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.302471ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
236842025-11-12T20:56:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
236852025-11-12T20:56:55.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236862025-11-12T20:56:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236872025-11-12T20:56:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
236882025-11-12T20:56:55.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
236892025-11-12T20:56:55.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
236902025-11-12T20:56:55.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
236912025-11-12T20:56:55.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.882124ms iteration = 137 status = {"config_updated":false}
236922025-11-12T20:56:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236932025-11-12T20:56:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236942025-11-12T20:56:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236952025-11-12T20:56:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
236962025-11-12T20:56:55.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236972025-11-12T20:56:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236982025-11-12T20:56:55.302ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
236992025-11-12T20:56:55.302ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.687259ms iteration = 69 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
237002025-11-12T20:56:57.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 287 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 9a9e41ca-d625-4f42-a2ab-1fd8f04a3078 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
237012025-11-12T20:57:00.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 138 reason = Timeout
237022025-11-12T20:57:00.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
237032025-11-12T20:57:00.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
237042025-11-12T20:57:00.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237052025-11-12T20:57:00.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
237062025-11-12T20:57:00.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
237072025-11-12T20:57:00.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
237082025-11-12T20:57:00.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
237092025-11-12T20:57:00.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.685023ms iteration = 138 status = {"config_updated":false}
237102025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
237112025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
237122025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
237132025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
237142025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
237152025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
237162025-11-12T20:57:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
237172025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 61 reason = Timeout
237182025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
237192025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
237202025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
237212025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
237222025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
237232025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
237242025-11-12T20:57:05.276ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
237252025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
237262025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
237272025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
237282025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
237292025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
237302025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
237312025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
237322025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
237332025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 36 reason = Timeout
237342025-11-12T20:57:05.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
237352025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
237362025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
237372025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
237382025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
237392025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
237402025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 36 reason = Timeout
237412025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
237422025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 139 reason = Timeout
237432025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
237442025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
237452025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 39 reason = Timeout
237462025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
237472025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
237482025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
237492025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
237502025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
237512025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
237522025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
237532025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
237542025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
237552025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
237562025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
237572025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
237582025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
237592025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
237602025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
237612025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
237622025-11-12T20:57:05.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 774737665 } })
237632025-11-12T20:57:05.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 774766976 } })
237642025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
237652025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
237662025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
237672025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
237682025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
237692025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
237702025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
237712025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
237722025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
237732025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
237742025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
237752025-11-12T20:57:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
237762025-11-12T20:57:05.278ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.401814ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
237772025-11-12T20:57:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
237782025-11-12T20:57:05.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.385798ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
237792025-11-12T20:57:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237802025-11-12T20:57:05.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237812025-11-12T20:57:05.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
237822025-11-12T20:57:05.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237832025-11-12T20:57:05.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237842025-11-12T20:57:05.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237852025-11-12T20:57:05.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237862025-11-12T20:57:05.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237872025-11-12T20:57:05.282ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
237882025-11-12T20:57:05.282ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 779967111 } })
237892025-11-12T20:57:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
237902025-11-12T20:57:05.283ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
237912025-11-12T20:57:05.283ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
237922025-11-12T20:57:05.283ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
237932025-11-12T20:57:05.283ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
237942025-11-12T20:57:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
237952025-11-12T20:57:05.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 47 reason = Timeout
237962025-11-12T20:57:05.284ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237972025-11-12T20:57:05.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
237982025-11-12T20:57:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
237992025-11-12T20:57:05.285ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 8.077799ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
238002025-11-12T20:57:05.286ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
238012025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
238022025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
238032025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
238042025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
238052025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238062025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
238072025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
238082025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
238092025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
238102025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
238112025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
238122025-11-12T20:57:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
238132025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
238142025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
238152025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
238162025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
238172025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
238182025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
238192025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
238202025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
238212025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
238222025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
238232025-11-12T20:57:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
238242025-11-12T20:57:05.288ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
238252025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
238262025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
238272025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 11.930129ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
238282025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
238292025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
238302025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
238312025-11-12T20:57:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
238322025-11-12T20:57:05.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 47 reason = Timeout
238332025-11-12T20:57:05.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
238342025-11-12T20:57:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
238352025-11-12T20:57:05.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
238362025-11-12T20:57:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
238372025-11-12T20:57:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
238382025-11-12T20:57:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
238392025-11-12T20:57:05.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
238402025-11-12T20:57:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
238412025-11-12T20:57:05.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238422025-11-12T20:57:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
238432025-11-12T20:57:05.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238442025-11-12T20:57:05.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238452025-11-12T20:57:05.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238462025-11-12T20:57:05.297ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1620, tv_nsec: 794424855 } })
238472025-11-12T20:57:05.297ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
238482025-11-12T20:57:05.298ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 14.444663ms iteration = 47 status = "NoSitrep"
238492025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
238502025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
238512025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
238522025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
238532025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 22.267976ms iteration = 139 status = {"config_updated":false}
238542025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
238552025-11-12T20:57:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
238562025-11-12T20:57:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
238572025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
238582025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
238592025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
238602025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 98e4220e-c01e-4ccf-adc3-06f59cdfaa04 original_time_started = 2025-11-12 20:56:36.679787 UTC
238612025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 24.995777ms iteration = 61 status = {"Loaded":{"collection_id":"98e4220e-c01e-4ccf-adc3-06f59cdfaa04","time_loaded":"2025-11-12T20:57:05.276266405Z","time_started":"2025-11-12T20:56:36.679787Z"}}
238622025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
238632025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
238642025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
238652025-11-12T20:57:05.302ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
238662025-11-12T20:57:05.302ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 25.559411ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
238672025-11-12T20:57:05.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238682025-11-12T20:57:05.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
238692025-11-12T20:57:05.308ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
238702025-11-12T20:57:05.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
238712025-11-12T20:57:05.310ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
238722025-11-12T20:57:05.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
238732025-11-12T20:57:05.311ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
238742025-11-12T20:57:05.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
238752025-11-12T20:57:05.312ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 35.452287ms iteration = 24 status = 0
238762025-11-12T20:57:05.314ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238772025-11-12T20:57:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
238782025-11-12T20:57:05.318ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
238792025-11-12T20:57:05.318ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 28.127208ms iteration = 47 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
238802025-11-12T20:57:05.320ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 43.253869ms iteration = 39 status = {"physical_disks_added":0}
238812025-11-12T20:57:05.329ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
238822025-11-12T20:57:05.329ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 52.588561ms iteration = 70 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
238832025-11-12T20:57:05.351ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
238842025-11-12T20:57:05.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
238852025-11-12T20:57:05.351ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
238862025-11-12T20:57:05.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
238872025-11-12T20:57:05.354ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
238882025-11-12T20:57:05.354ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 77.427413ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
238892025-11-12T20:57:05.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
238902025-11-12T20:57:05.356ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
238912025-11-12T20:57:05.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
238922025-11-12T20:57:05.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 80.470073ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
238932025-11-12T20:57:05.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238942025-11-12T20:57:05.360ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 83.08446ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
238952025-11-12T20:57:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
238962025-11-12T20:57:05.362ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
238972025-11-12T20:57:05.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
238982025-11-12T20:57:05.364ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
238992025-11-12T20:57:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
239002025-11-12T20:57:05.365ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
239012025-11-12T20:57:05.365ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 88.548091ms iteration = 36 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
239022025-11-12T20:57:05.373ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 96.401995ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
239032025-11-12T20:57:05.376ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
239042025-11-12T20:57:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
239052025-11-12T20:57:05.381ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
239062025-11-12T20:57:05.381ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
239072025-11-12T20:57:05.381ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 104.034032ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
239082025-11-12T20:57:05.382ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239092025-11-12T20:57:05.385ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
239102025-11-12T20:57:05.385ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
239112025-11-12T20:57:05.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
239122025-11-12T20:57:05.387ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239132025-11-12T20:57:05.391ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
239142025-11-12T20:57:05.391ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
239152025-11-12T20:57:05.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
239162025-11-12T20:57:05.417ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
239172025-11-12T20:57:05.417ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
239182025-11-12T20:57:05.417ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b563a3b1-412e-43e9-8c7d-939721616b3e", "content-length": "26", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239192025-11-12T20:57:05.417ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
239202025-11-12T20:57:05.420ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
239212025-11-12T20:57:05.431ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
239222025-11-12T20:57:05.431ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
239232025-11-12T20:57:05.431ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "823f424b-0692-4ff1-ba68-11e6037a789a", "content-length": "26", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239242025-11-12T20:57:05.431ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
239252025-11-12T20:57:05.433ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
239262025-11-12T20:57:05.433ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
239272025-11-12T20:57:05.434ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
239282025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
239292025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "e22d9403-56d0-415b-8ce4-2738ab13a598", "content-length": "78", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239302025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
239312025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
239322025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
239332025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
239342025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
239352025-11-12T20:57:05.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
239362025-11-12T20:57:05.438ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
239372025-11-12T20:57:05.438ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
239382025-11-12T20:57:05.438ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
239392025-11-12T20:57:05.439ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
239402025-11-12T20:57:05.439ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b9f3b90-23f3-4412-a457-e77fe34ca2a9", "content-length": "372", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239412025-11-12T20:57:05.439ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239422025-11-12T20:57:05.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239432025-11-12T20:57:05.441ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 167.083424ms iteration = 24 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
239442025-11-12T20:57:05.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
239452025-11-12T20:57:05.442ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
239462025-11-12T20:57:05.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
239472025-11-12T20:57:05.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
239482025-11-12T20:57:05.442ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
239492025-11-12T20:57:05.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
239502025-11-12T20:57:05.443ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
239512025-11-12T20:57:05.443ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
239522025-11-12T20:57:05.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
239532025-11-12T20:57:05.444ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
239542025-11-12T20:57:05.444ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
239552025-11-12T20:57:05.444ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f7ae9e5-7e82-4ef7-9d79-780ac7267b33", "content-length": "81", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239562025-11-12T20:57:05.445ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
239572025-11-12T20:57:05.445ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
239582025-11-12T20:57:05.445ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "709c8fec-eab6-48c6-a94b-9ac9a587d2f0", "content-length": "81", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239592025-11-12T20:57:05.445ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
239602025-11-12T20:57:05.445ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
239612025-11-12T20:57:05.445ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "4451377e-b79a-4994-93d1-0283888544d3", "content-length": "81", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239622025-11-12T20:57:05.447ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 170.322528ms iteration = 24 status = {"disabled":false,"errors":[],"sps":[]}
239632025-11-12T20:57:05.490ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
239642025-11-12T20:57:05.490ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
239652025-11-12T20:57:05.490ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "11bb59de-3830-4ea7-ba82-a533ee0783b7", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239662025-11-12T20:57:05.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239672025-11-12T20:57:05.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239682025-11-12T20:57:05.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239692025-11-12T20:57:05.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239702025-11-12T20:57:05.494ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239712025-11-12T20:57:05.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239722025-11-12T20:57:05.496ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
239732025-11-12T20:57:05.496ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cedd995-7443-4c97-b47d-7400fb0c7b60", "content-length": "691", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239742025-11-12T20:57:05.496ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
239752025-11-12T20:57:05.496ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
239762025-11-12T20:57:05.496ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
239772025-11-12T20:57:05.497ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
239782025-11-12T20:57:05.497ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
239792025-11-12T20:57:05.498ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8b3ffc89-a0b2-46ad-a434-263db688dfd5", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
239802025-11-12T20:57:05.499ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 222.092387ms iteration = 24 status = {}
239812025-11-12T20:57:05.500ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 223.930395ms iteration = 36 status = {}
239822025-11-12T20:57:05.505ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1621, tv_nsec: 2706185 } })
239832025-11-12T20:57:05.515ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 241.740205ms iteration = 24 status = {}
239842025-11-12T20:57:05.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
239852025-11-12T20:57:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
239862025-11-12T20:57:05.573ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
239872025-11-12T20:57:05.573ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
239882025-11-12T20:57:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
239892025-11-12T20:57:05.582ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
239902025-11-12T20:57:05.582ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
239912025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
239922025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
239932025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239942025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239952025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239962025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
239972025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239982025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
239992025-11-12T20:57:05.583ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
240002025-11-12T20:57:05.583ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
240012025-11-12T20:57:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
240022025-11-12T20:57:05.586ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
240032025-11-12T20:57:05.587ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
240042025-11-12T20:57:05.697ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "71cb15be-9157-4b4a-8861-e2aa096768fd", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
240052025-11-12T20:57:05.697ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
240062025-11-12T20:57:05.697ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
240072025-11-12T20:57:05.703ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b99e4c6-fc4c-4649-85a2-9bc9c4523805", "content-length": "682", "date": "Wed, 12 Nov 2025 20:57:05 GMT"} })
240082025-11-12T20:57:05.703ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
240092025-11-12T20:57:05.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
240102025-11-12T20:57:05.707ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
240112025-11-12T20:57:05.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
240122025-11-12T20:57:05.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
240132025-11-12T20:57:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
240142025-11-12T20:57:05.713ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
240152025-11-12T20:57:05.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
240162025-11-12T20:57:05.720ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 443.690293ms iteration = 24 status = {}
240172025-11-12T20:57:07.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 429 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 7c467376-b360-4390-b5b0-7059ab47ebe6 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
240182025-11-12T20:57:10.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 140 reason = Timeout
240192025-11-12T20:57:10.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
240202025-11-12T20:57:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
240212025-11-12T20:57:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240222025-11-12T20:57:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
240232025-11-12T20:57:10.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
240242025-11-12T20:57:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
240252025-11-12T20:57:10.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
240262025-11-12T20:57:10.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.845478ms iteration = 140 status = {"config_updated":false}
240272025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
240282025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
240292025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
240302025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
240312025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 141 reason = Timeout
240322025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
240332025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
240342025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
240352025-11-12T20:57:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
240362025-11-12T20:57:15.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
240372025-11-12T20:57:15.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240382025-11-12T20:57:15.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
240392025-11-12T20:57:15.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240402025-11-12T20:57:15.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.529062ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
240412025-11-12T20:57:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
240422025-11-12T20:57:15.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
240432025-11-12T20:57:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
240442025-11-12T20:57:15.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
240452025-11-12T20:57:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
240462025-11-12T20:57:15.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
240472025-11-12T20:57:15.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
240482025-11-12T20:57:15.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
240492025-11-12T20:57:15.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.155916ms iteration = 141 status = {"config_updated":false}
240502025-11-12T20:57:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
240512025-11-12T20:57:15.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240522025-11-12T20:57:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
240532025-11-12T20:57:15.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
240542025-11-12T20:57:15.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
240552025-11-12T20:57:15.301ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
240562025-11-12T20:57:15.301ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.400101ms iteration = 71 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
240572025-11-12T20:57:17.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 302 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = e23776fc-183d-4b1b-9f1b-7f9153686c36 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
240582025-11-12T20:57:20.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 62 reason = Timeout
240592025-11-12T20:57:20.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
240602025-11-12T20:57:20.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
240612025-11-12T20:57:20.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 142 reason = Timeout
240622025-11-12T20:57:20.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
240632025-11-12T20:57:20.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
240642025-11-12T20:57:20.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240652025-11-12T20:57:20.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240662025-11-12T20:57:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
240672025-11-12T20:57:20.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
240682025-11-12T20:57:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
240692025-11-12T20:57:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
240702025-11-12T20:57:20.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
240712025-11-12T20:57:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
240722025-11-12T20:57:20.284ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 98e4220e-c01e-4ccf-adc3-06f59cdfaa04 original_time_started = 2025-11-12 20:56:36.679787 UTC
240732025-11-12T20:57:20.284ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.160622ms iteration = 62 status = {"Loaded":{"collection_id":"98e4220e-c01e-4ccf-adc3-06f59cdfaa04","time_loaded":"2025-11-12T20:57:20.276051143Z","time_started":"2025-11-12T20:56:36.679787Z"}}
240742025-11-12T20:57:20.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 48 reason = Timeout
240752025-11-12T20:57:20.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
240762025-11-12T20:57:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
240772025-11-12T20:57:20.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
240782025-11-12T20:57:20.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.240143ms iteration = 142 status = {"config_updated":false}
240792025-11-12T20:57:20.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240802025-11-12T20:57:20.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
240812025-11-12T20:57:20.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
240822025-11-12T20:57:20.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
240832025-11-12T20:57:20.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 48 reason = Timeout
240842025-11-12T20:57:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
240852025-11-12T20:57:20.290ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.16952ms iteration = 48 status = "NoSitrep"
240862025-11-12T20:57:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
240872025-11-12T20:57:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
240882025-11-12T20:57:20.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
240892025-11-12T20:57:20.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240902025-11-12T20:57:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
240912025-11-12T20:57:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
240922025-11-12T20:57:20.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
240932025-11-12T20:57:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
240942025-11-12T20:57:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
240952025-11-12T20:57:20.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240962025-11-12T20:57:20.300ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
240972025-11-12T20:57:20.300ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
240982025-11-12T20:57:20.300ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.794059ms iteration = 48 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
240992025-11-12T20:57:25.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
241002025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
241012025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
241022025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
241032025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 143 reason = Timeout
241042025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
241052025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
241062025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
241072025-11-12T20:57:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
241082025-11-12T20:57:25.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241092025-11-12T20:57:25.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
241102025-11-12T20:57:25.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
241112025-11-12T20:57:25.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241122025-11-12T20:57:25.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.534051ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
241132025-11-12T20:57:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
241142025-11-12T20:57:25.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
241152025-11-12T20:57:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
241162025-11-12T20:57:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
241172025-11-12T20:57:25.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
241182025-11-12T20:57:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
241192025-11-12T20:57:25.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
241202025-11-12T20:57:25.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
241212025-11-12T20:57:25.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.007377ms iteration = 143 status = {"config_updated":false}
241222025-11-12T20:57:25.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
241232025-11-12T20:57:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241242025-11-12T20:57:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
241252025-11-12T20:57:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
241262025-11-12T20:57:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
241272025-11-12T20:57:25.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
241282025-11-12T20:57:25.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.80288ms iteration = 72 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
241292025-11-12T20:57:27.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 283 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = ab7f4630-e8ad-45c5-81e2-e0ece0ea4073 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
241302025-11-12T20:57:30.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 144 reason = Timeout
241312025-11-12T20:57:30.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
241322025-11-12T20:57:30.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
241332025-11-12T20:57:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241342025-11-12T20:57:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
241352025-11-12T20:57:30.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
241362025-11-12T20:57:30.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
241372025-11-12T20:57:30.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
241382025-11-12T20:57:30.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.905718ms iteration = 144 status = {"config_updated":false}
241392025-11-12T20:57:32.914ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:50720
241402025-11-12T20:57:32.914ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 1764cce5-fcb2-41fb-b3f0-5691cf59b397 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
241412025-11-12T20:57:32.914ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 1764cce5-fcb2-41fb-b3f0-5691cf59b397 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
241422025-11-12T20:57:32.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 1764cce5-fcb2-41fb-b3f0-5691cf59b397 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
241432025-11-12T20:57:32.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3415 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 1764cce5-fcb2-41fb-b3f0-5691cf59b397 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
241442025-11-12T20:57:32.918ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = e82e051b-a3d4-46aa-9392-ff169addcd6e uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
241452025-11-12T20:57:32.918ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = e82e051b-a3d4-46aa-9392-ff169addcd6e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
241462025-11-12T20:57:32.918ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = e82e051b-a3d4-46aa-9392-ff169addcd6e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
241472025-11-12T20:57:32.920ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2496 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = e82e051b-a3d4-46aa-9392-ff169addcd6e response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
241482025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
241492025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
241502025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
241512025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
241522025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
241532025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
241542025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
241552025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
241562025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
241572025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
241582025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
241592025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
241602025-11-12T20:57:35.273ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
241612025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
241622025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
241632025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
241642025-11-12T20:57:35.273ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = ExternalNetworking step id = Ensure
241652025-11-12T20:57:35.273ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:47:35.273888795 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
241662025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
241672025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
241682025-11-12T20:57:35.273ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
241692025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241702025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
241712025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
241722025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
241732025-11-12T20:57:35.274ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 771756203 } })
241742025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
241752025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
241762025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
241772025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
241782025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
241792025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
241802025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241812025-11-12T20:57:35.274ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 772143153 } })
241822025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
241832025-11-12T20:57:35.274ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.241262ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
241842025-11-12T20:57:35.275ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
241852025-11-12T20:57:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.733725ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
241862025-11-12T20:57:35.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 63 reason = Timeout
241872025-11-12T20:57:35.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
241882025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
241892025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
241902025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
241912025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
241922025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 40 reason = Timeout
241932025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
241942025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
241952025-11-12T20:57:35.276ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
241962025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
241972025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
241982025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
241992025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
242002025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
242012025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
242022025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
242032025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
242042025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
242052025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
242062025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
242072025-11-12T20:57:35.276ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
242082025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
242092025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
242102025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
242112025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
242122025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 145 reason = Timeout
242132025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
242142025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
242152025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
242162025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
242172025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
242182025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
242192025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
242202025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
242212025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
242222025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
242232025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
242242025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
242252025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
242262025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
242272025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
242282025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
242292025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
242302025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
242312025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
242322025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
242332025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
242342025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
242352025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 37 reason = Timeout
242362025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
242372025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 30 reason = Timeout
242382025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
242392025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
242402025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242412025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
242422025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
242432025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
242442025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
242452025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
242462025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
242472025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
242482025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 37 reason = Timeout
242492025-11-12T20:57:35.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
242502025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
242512025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
242522025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
242532025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
242542025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
242552025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
242562025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
242572025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
242582025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
242592025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
242602025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
242612025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
242622025-11-12T20:57:35.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 775235263 } })
242632025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
242642025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242652025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
242662025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
242672025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
242682025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
242692025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
242702025-11-12T20:57:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
242712025-11-12T20:57:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
242722025-11-12T20:57:35.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
242732025-11-12T20:57:35.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242742025-11-12T20:57:35.278ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242752025-11-12T20:57:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242762025-11-12T20:57:35.278ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:47:35.273888795 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
242772025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.082441ms iteration = 13 status = {"expiration":"2025-11-12T20:47:35.273888795Z","pruned":[]}
242782025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
242792025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242802025-11-12T20:57:35.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 776567098 } })
242812025-11-12T20:57:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 776608639 } })
242822025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
242832025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
242842025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242852025-11-12T20:57:35.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
242862025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
242872025-11-12T20:57:35.279ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
242882025-11-12T20:57:35.280ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242892025-11-12T20:57:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
242902025-11-12T20:57:35.280ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
242912025-11-12T20:57:35.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242922025-11-12T20:57:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
242932025-11-12T20:57:35.280ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242942025-11-12T20:57:35.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242952025-11-12T20:57:35.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.2406ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
242962025-11-12T20:57:35.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242972025-11-12T20:57:35.281ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.674588ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
242982025-11-12T20:57:35.281ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
242992025-11-12T20:57:35.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243002025-11-12T20:57:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
243012025-11-12T20:57:35.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
243022025-11-12T20:57:35.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 49 reason = Timeout
243032025-11-12T20:57:35.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
243042025-11-12T20:57:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
243052025-11-12T20:57:35.284ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243062025-11-12T20:57:35.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
243072025-11-12T20:57:35.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243082025-11-12T20:57:35.285ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
243092025-11-12T20:57:35.285ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
243102025-11-12T20:57:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
243112025-11-12T20:57:35.286ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
243122025-11-12T20:57:35.286ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
243132025-11-12T20:57:35.286ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 9.239619ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
243142025-11-12T20:57:35.286ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243152025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
243162025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
243172025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.190034ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
243182025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
243192025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
243202025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
243212025-11-12T20:57:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
243222025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
243232025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
243242025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
243252025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
243262025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
243272025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
243282025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
243292025-11-12T20:57:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
243302025-11-12T20:57:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
243312025-11-12T20:57:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
243322025-11-12T20:57:35.289ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
243332025-11-12T20:57:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
243342025-11-12T20:57:35.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
243352025-11-12T20:57:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
243362025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 13 reason = Timeout
243372025-11-12T20:57:35.290ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
243382025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
243392025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
243402025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 49 reason = Timeout
243412025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
243422025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
243432025-11-12T20:57:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
243442025-11-12T20:57:35.291ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
243452025-11-12T20:57:35.291ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 14.148636ms iteration = 13 status = {"by_rx":{},"error":null}
243462025-11-12T20:57:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
243472025-11-12T20:57:35.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
243482025-11-12T20:57:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
243492025-11-12T20:57:35.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243502025-11-12T20:57:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
243512025-11-12T20:57:35.292ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
243522025-11-12T20:57:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
243532025-11-12T20:57:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
243542025-11-12T20:57:35.293ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
243552025-11-12T20:57:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
243562025-11-12T20:57:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
243572025-11-12T20:57:35.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
243582025-11-12T20:57:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
243592025-11-12T20:57:35.295ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243602025-11-12T20:57:35.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
243612025-11-12T20:57:35.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
243622025-11-12T20:57:35.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
243632025-11-12T20:57:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
243642025-11-12T20:57:35.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
243652025-11-12T20:57:35.296ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243662025-11-12T20:57:35.297ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
243672025-11-12T20:57:35.297ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
243682025-11-12T20:57:35.297ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
243692025-11-12T20:57:35.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
243702025-11-12T20:57:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
243712025-11-12T20:57:35.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243722025-11-12T20:57:35.299ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
243732025-11-12T20:57:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
243742025-11-12T20:57:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
243752025-11-12T20:57:35.299ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
243762025-11-12T20:57:35.299ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 23.027926ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
243772025-11-12T20:57:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
243782025-11-12T20:57:35.300ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
243792025-11-12T20:57:35.300ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
243802025-11-12T20:57:35.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
243812025-11-12T20:57:35.300ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243822025-11-12T20:57:35.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243832025-11-12T20:57:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
243842025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 98e4220e-c01e-4ccf-adc3-06f59cdfaa04 original_time_started = 2025-11-12 20:56:36.679787 UTC
243852025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 26.459315ms iteration = 63 status = {"Loaded":{"collection_id":"98e4220e-c01e-4ccf-adc3-06f59cdfaa04","time_loaded":"2025-11-12T20:57:35.275705302Z","time_started":"2025-11-12T20:56:36.679787Z"}}
243862025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
243872025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
243882025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
243892025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
243902025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
243912025-11-12T20:57:35.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
243922025-11-12T20:57:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
243932025-11-12T20:57:35.303ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
243942025-11-12T20:57:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
243952025-11-12T20:57:35.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
243962025-11-12T20:57:35.304ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243972025-11-12T20:57:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
243982025-11-12T20:57:35.304ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
243992025-11-12T20:57:35.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
244002025-11-12T20:57:35.305ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244012025-11-12T20:57:35.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
244022025-11-12T20:57:35.305ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
244032025-11-12T20:57:35.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
244042025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
244052025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
244062025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
244072025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 31.979278ms iteration = 145 status = {"config_updated":false}
244082025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
244092025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
244102025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
244112025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
244122025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
244132025-11-12T20:57:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
244142025-11-12T20:57:35.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244152025-11-12T20:57:35.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 35.844107ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
244162025-11-12T20:57:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
244172025-11-12T20:57:35.316ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244182025-11-12T20:57:35.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244192025-11-12T20:57:35.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244202025-11-12T20:57:35.318ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1650, tv_nsec: 815753352 } })
244212025-11-12T20:57:35.318ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
244222025-11-12T20:57:35.320ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 43.288ms iteration = 25 status = 0
244232025-11-12T20:57:35.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244242025-11-12T20:57:35.320ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
244252025-11-12T20:57:35.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244262025-11-12T20:57:35.322ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 38.078885ms iteration = 49 status = "NoSitrep"
244272025-11-12T20:57:35.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244282025-11-12T20:57:35.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244292025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244302025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
244312025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
244322025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244332025-11-12T20:57:35.323ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
244342025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
244352025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
244362025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
244372025-11-12T20:57:35.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
244382025-11-12T20:57:35.324ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
244392025-11-12T20:57:35.324ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
244402025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
244412025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
244422025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
244432025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
244442025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
244452025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
244462025-11-12T20:57:35.325ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 48.729231ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
244472025-11-12T20:57:35.326ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
244482025-11-12T20:57:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244492025-11-12T20:57:35.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
244502025-11-12T20:57:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244512025-11-12T20:57:35.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
244522025-11-12T20:57:35.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
244532025-11-12T20:57:35.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
244542025-11-12T20:57:35.328ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
244552025-11-12T20:57:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
244562025-11-12T20:57:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
244572025-11-12T20:57:35.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244582025-11-12T20:57:35.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244592025-11-12T20:57:35.331ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
244602025-11-12T20:57:35.333ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
244612025-11-12T20:57:35.333ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
244622025-11-12T20:57:35.333ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
244632025-11-12T20:57:35.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
244642025-11-12T20:57:35.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244652025-11-12T20:57:35.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244662025-11-12T20:57:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244672025-11-12T20:57:35.336ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
244682025-11-12T20:57:35.338ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
244692025-11-12T20:57:35.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
244702025-11-12T20:57:35.339ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
244712025-11-12T20:57:35.340ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
244722025-11-12T20:57:35.340ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 49.860001ms iteration = 49 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
244732025-11-12T20:57:35.344ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
244742025-11-12T20:57:35.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244752025-11-12T20:57:35.345ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244762025-11-12T20:57:35.347ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
244772025-11-12T20:57:35.348ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244782025-11-12T20:57:35.348ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 71.790058ms iteration = 40 status = {"physical_disks_added":0}
244792025-11-12T20:57:35.349ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
244802025-11-12T20:57:35.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
244812025-11-12T20:57:35.350ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
244822025-11-12T20:57:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
244832025-11-12T20:57:35.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244842025-11-12T20:57:35.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244852025-11-12T20:57:35.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244862025-11-12T20:57:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
244872025-11-12T20:57:35.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
244882025-11-12T20:57:35.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
244892025-11-12T20:57:35.355ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
244902025-11-12T20:57:35.358ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
244912025-11-12T20:57:35.361ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
244922025-11-12T20:57:35.364ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
244932025-11-12T20:57:35.365ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = SledList step id = Fetch
244942025-11-12T20:57:35.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
244952025-11-12T20:57:35.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
244962025-11-12T20:57:35.367ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
244972025-11-12T20:57:35.367ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
244982025-11-12T20:57:35.367ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
244992025-11-12T20:57:35.367ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 93.448458ms iteration = 13 status = {"generation":2}
245002025-11-12T20:57:35.368ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
245012025-11-12T20:57:35.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
245022025-11-12T20:57:35.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245032025-11-12T20:57:35.371ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 93.882599ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
245042025-11-12T20:57:35.373ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
245052025-11-12T20:57:35.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
245062025-11-12T20:57:35.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
245072025-11-12T20:57:35.375ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
245082025-11-12T20:57:35.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
245092025-11-12T20:57:35.378ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245102025-11-12T20:57:35.379ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.711158ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
245112025-11-12T20:57:35.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245122025-11-12T20:57:35.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
245132025-11-12T20:57:35.383ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
245142025-11-12T20:57:35.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
245152025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
245162025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
245172025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
245182025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
245192025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
245202025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
245212025-11-12T20:57:35.385ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
245222025-11-12T20:57:35.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
245232025-11-12T20:57:35.389ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 112.227195ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
245242025-11-12T20:57:35.389ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 112.590983ms iteration = 6 status = {"nsuccess":5}
245252025-11-12T20:57:35.391ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245262025-11-12T20:57:35.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = DeployNexusRecords step id = Ensure
245272025-11-12T20:57:35.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
245282025-11-12T20:57:35.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
245292025-11-12T20:57:35.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245302025-11-12T20:57:35.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245312025-11-12T20:57:35.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
245322025-11-12T20:57:35.397ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
245332025-11-12T20:57:35.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
245342025-11-12T20:57:35.399ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
245352025-11-12T20:57:35.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
245362025-11-12T20:57:35.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
245372025-11-12T20:57:35.401ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
245382025-11-12T20:57:35.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
245392025-11-12T20:57:35.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
245402025-11-12T20:57:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
245412025-11-12T20:57:35.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
245422025-11-12T20:57:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
245432025-11-12T20:57:35.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
245442025-11-12T20:57:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245452025-11-12T20:57:35.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
245462025-11-12T20:57:35.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
245472025-11-12T20:57:35.407ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245482025-11-12T20:57:35.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245492025-11-12T20:57:35.410ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
245502025-11-12T20:57:35.410ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
245512025-11-12T20:57:35.410ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 133.732041ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
245522025-11-12T20:57:35.411ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 134.029558ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
245532025-11-12T20:57:35.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
245542025-11-12T20:57:35.411ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
245552025-11-12T20:57:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
245562025-11-12T20:57:35.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
245572025-11-12T20:57:35.414ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = SledAgent step id = Ensure
245582025-11-12T20:57:35.414ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
245592025-11-12T20:57:35.414ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
245602025-11-12T20:57:35.414ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 137.5473ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
245612025-11-12T20:57:35.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
245622025-11-12T20:57:35.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
245632025-11-12T20:57:35.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
245642025-11-12T20:57:35.422ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
245652025-11-12T20:57:35.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
245662025-11-12T20:57:35.425ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245672025-11-12T20:57:35.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
245682025-11-12T20:57:35.430ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
245692025-11-12T20:57:35.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
245702025-11-12T20:57:35.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245712025-11-12T20:57:35.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245722025-11-12T20:57:35.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245732025-11-12T20:57:35.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245742025-11-12T20:57:35.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245752025-11-12T20:57:35.440ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
245762025-11-12T20:57:35.440ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 163.662545ms iteration = 73 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
245772025-11-12T20:57:35.445ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
245782025-11-12T20:57:35.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245792025-11-12T20:57:35.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245802025-11-12T20:57:35.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245812025-11-12T20:57:35.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245822025-11-12T20:57:35.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245832025-11-12T20:57:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
245842025-11-12T20:57:35.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245852025-11-12T20:57:35.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245862025-11-12T20:57:35.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245872025-11-12T20:57:35.463ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
245882025-11-12T20:57:35.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245892025-11-12T20:57:35.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245902025-11-12T20:57:35.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245912025-11-12T20:57:35.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245922025-11-12T20:57:35.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245932025-11-12T20:57:35.471ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
245942025-11-12T20:57:35.471ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
245952025-11-12T20:57:35.471ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
245962025-11-12T20:57:35.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
245972025-11-12T20:57:35.473ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
245982025-11-12T20:57:35.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245992025-11-12T20:57:35.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246002025-11-12T20:57:35.476ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246012025-11-12T20:57:35.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246022025-11-12T20:57:35.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246032025-11-12T20:57:35.479ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
246042025-11-12T20:57:35.479ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 201.873985ms iteration = 37 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
246052025-11-12T20:57:35.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246062025-11-12T20:57:35.481ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
246072025-11-12T20:57:35.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
246082025-11-12T20:57:35.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
246092025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
246102025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246112025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
246122025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
246132025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
246142025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 212.842018ms iteration = 13 status = {"generation":1}
246152025-11-12T20:57:35.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246162025-11-12T20:57:35.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246172025-11-12T20:57:35.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246182025-11-12T20:57:35.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246192025-11-12T20:57:35.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
246202025-11-12T20:57:35.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246212025-11-12T20:57:35.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246222025-11-12T20:57:35.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246232025-11-12T20:57:35.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246242025-11-12T20:57:35.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246252025-11-12T20:57:35.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246262025-11-12T20:57:35.517ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
246272025-11-12T20:57:35.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
246282025-11-12T20:57:35.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246292025-11-12T20:57:35.521ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
246302025-11-12T20:57:35.521ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 247.31505ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
246312025-11-12T20:57:35.522ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
246322025-11-12T20:57:35.522ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 245.129334ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
246332025-11-12T20:57:35.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
246342025-11-12T20:57:35.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246352025-11-12T20:57:35.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246362025-11-12T20:57:35.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246372025-11-12T20:57:35.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246382025-11-12T20:57:35.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246392025-11-12T20:57:35.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
246402025-11-12T20:57:35.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246412025-11-12T20:57:35.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246422025-11-12T20:57:35.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246432025-11-12T20:57:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246442025-11-12T20:57:35.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246452025-11-12T20:57:35.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246462025-11-12T20:57:35.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246472025-11-12T20:57:35.539ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246482025-11-12T20:57:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246492025-11-12T20:57:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
246502025-11-12T20:57:35.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246512025-11-12T20:57:35.541ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246522025-11-12T20:57:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246532025-11-12T20:57:35.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246542025-11-12T20:57:35.544ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
246552025-11-12T20:57:35.544ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246562025-11-12T20:57:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246572025-11-12T20:57:35.546ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246582025-11-12T20:57:35.548ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
246592025-11-12T20:57:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246602025-11-12T20:57:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246612025-11-12T20:57:35.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
246622025-11-12T20:57:35.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246632025-11-12T20:57:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246642025-11-12T20:57:35.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246652025-11-12T20:57:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246662025-11-12T20:57:35.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246672025-11-12T20:57:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246682025-11-12T20:57:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246692025-11-12T20:57:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
246702025-11-12T20:57:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246712025-11-12T20:57:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246722025-11-12T20:57:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246732025-11-12T20:57:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246742025-11-12T20:57:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246752025-11-12T20:57:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
246762025-11-12T20:57:35.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246772025-11-12T20:57:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246782025-11-12T20:57:35.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246792025-11-12T20:57:35.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246802025-11-12T20:57:35.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246812025-11-12T20:57:35.577ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
246822025-11-12T20:57:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
246832025-11-12T20:57:35.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246842025-11-12T20:57:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246852025-11-12T20:57:35.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246862025-11-12T20:57:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246872025-11-12T20:57:35.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246882025-11-12T20:57:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
246892025-11-12T20:57:35.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246902025-11-12T20:57:35.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246912025-11-12T20:57:35.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246922025-11-12T20:57:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246932025-11-12T20:57:35.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246942025-11-12T20:57:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
246952025-11-12T20:57:35.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246962025-11-12T20:57:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246972025-11-12T20:57:35.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
246982025-11-12T20:57:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
246992025-11-12T20:57:35.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247002025-11-12T20:57:35.600ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
247012025-11-12T20:57:35.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
247022025-11-12T20:57:35.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
247032025-11-12T20:57:35.602ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
247042025-11-12T20:57:35.602ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
247052025-11-12T20:57:35.603ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
247062025-11-12T20:57:35.603ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
247072025-11-12T20:57:35.603ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a5ee1dac-ab05-4a04-b452-30de0dc9e757 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
247082025-11-12T20:57:35.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
247092025-11-12T20:57:35.603ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e4b1f0e-48a1-445e-9a7d-e19b19f18634", "content-length": "26", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
247102025-11-12T20:57:35.603ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
247112025-11-12T20:57:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
247122025-11-12T20:57:35.605ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
247132025-11-12T20:57:35.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247142025-11-12T20:57:35.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
247152025-11-12T20:57:35.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
247162025-11-12T20:57:35.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
247172025-11-12T20:57:35.613ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
247182025-11-12T20:57:35.613ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
247192025-11-12T20:57:35.613ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 336.18853ms iteration = 30 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
247202025-11-12T20:57:35.644ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
247212025-11-12T20:57:35.644ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
247222025-11-12T20:57:35.644ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce05ca5c-6c15-4875-9f8d-f8fd9e041748", "content-length": "26", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
247232025-11-12T20:57:35.644ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
247242025-11-12T20:57:35.646ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
247252025-11-12T20:57:35.712ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
247262025-11-12T20:57:35.714ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "4139c26b-8448-4af8-8c64-0434671240e5", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
247272025-11-12T20:57:35.715ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = FirewallRules step id = Ensure
247282025-11-12T20:57:35.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247292025-11-12T20:57:35.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247302025-11-12T20:57:35.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247312025-11-12T20:57:35.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247322025-11-12T20:57:35.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247332025-11-12T20:57:35.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247342025-11-12T20:57:35.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247352025-11-12T20:57:35.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247362025-11-12T20:57:35.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247372025-11-12T20:57:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247382025-11-12T20:57:35.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247392025-11-12T20:57:35.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247402025-11-12T20:57:35.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247412025-11-12T20:57:35.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
247422025-11-12T20:57:35.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247432025-11-12T20:57:35.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247442025-11-12T20:57:35.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247452025-11-12T20:57:35.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247462025-11-12T20:57:35.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247472025-11-12T20:57:35.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247482025-11-12T20:57:35.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247492025-11-12T20:57:35.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
247502025-11-12T20:57:35.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247512025-11-12T20:57:35.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247522025-11-12T20:57:35.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247532025-11-12T20:57:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247542025-11-12T20:57:35.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247552025-11-12T20:57:35.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247562025-11-12T20:57:35.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247572025-11-12T20:57:35.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247582025-11-12T20:57:35.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247592025-11-12T20:57:35.758ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
247602025-11-12T20:57:35.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247612025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "96a8d07f-4aa1-41fb-95c9-8f7569c69532", "content-length": "78", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
247622025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
247632025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
247642025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
247652025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
247662025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
247672025-11-12T20:57:35.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
247682025-11-12T20:57:35.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247692025-11-12T20:57:35.761ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
247702025-11-12T20:57:35.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247712025-11-12T20:57:35.761ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
247722025-11-12T20:57:35.761ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
247732025-11-12T20:57:35.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247742025-11-12T20:57:35.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247752025-11-12T20:57:35.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
247762025-11-12T20:57:35.764ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
247772025-11-12T20:57:35.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
247782025-11-12T20:57:35.765ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
247792025-11-12T20:57:35.765ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
247802025-11-12T20:57:35.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247812025-11-12T20:57:35.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
247822025-11-12T20:57:35.765ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
247832025-11-12T20:57:35.765ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
247842025-11-12T20:57:35.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
247852025-11-12T20:57:35.766ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
247862025-11-12T20:57:35.766ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
247872025-11-12T20:57:35.766ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f34fa729-7efb-4b81-b9d3-54e96e50ed54", "content-length": "81", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
247882025-11-12T20:57:35.767ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
247892025-11-12T20:57:35.767ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
247902025-11-12T20:57:35.767ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
247912025-11-12T20:57:35.767ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f725a810-2d86-49c9-9e94-44c027437b27", "content-length": "81", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
247922025-11-12T20:57:35.768ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
247932025-11-12T20:57:35.768ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
247942025-11-12T20:57:35.768ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "137e608b-535d-401d-9f28-30b826cf9ee5", "content-length": "81", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
247952025-11-12T20:57:35.769ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 495.675568ms iteration = 25 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
247962025-11-12T20:57:35.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
247972025-11-12T20:57:35.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
247982025-11-12T20:57:35.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
247992025-11-12T20:57:35.770ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 493.376188ms iteration = 25 status = {"disabled":false,"errors":[],"sps":[]}
248002025-11-12T20:57:35.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248012025-11-12T20:57:35.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248022025-11-12T20:57:35.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248032025-11-12T20:57:35.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248042025-11-12T20:57:35.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248052025-11-12T20:57:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248062025-11-12T20:57:35.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248072025-11-12T20:57:35.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248082025-11-12T20:57:35.779ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
248092025-11-12T20:57:35.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248102025-11-12T20:57:35.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248112025-11-12T20:57:35.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248122025-11-12T20:57:35.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248132025-11-12T20:57:35.782ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
248142025-11-12T20:57:35.782ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
248152025-11-12T20:57:35.783ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "26d4f9ab-d321-4260-aac3-5d4dd29a7a40", "content-length": "523", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
248162025-11-12T20:57:35.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248172025-11-12T20:57:35.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248182025-11-12T20:57:35.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248192025-11-12T20:57:35.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
248202025-11-12T20:57:35.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248212025-11-12T20:57:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248222025-11-12T20:57:35.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248232025-11-12T20:57:35.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248242025-11-12T20:57:35.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248252025-11-12T20:57:35.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248262025-11-12T20:57:35.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248272025-11-12T20:57:35.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
248282025-11-12T20:57:35.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248292025-11-12T20:57:35.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248302025-11-12T20:57:35.802ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {494f8547-7581-4a15-8580-749d6dba22ca: {10.151.2.176: {a82ec15b-487e-4425-80d7-493e728f3e39}, 10.151.2.175: {a82ec15b-487e-4425-80d7-493e728f3e39}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
248312025-11-12T20:57:35.802ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
248322025-11-12T20:57:35.802ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "445974fa-c7f9-4970-aa47-2665e8ca21a3", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
248332025-11-12T20:57:35.802ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248342025-11-12T20:57:35.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248352025-11-12T20:57:35.805ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248362025-11-12T20:57:35.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248372025-11-12T20:57:35.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248382025-11-12T20:57:35.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248392025-11-12T20:57:35.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248402025-11-12T20:57:35.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248412025-11-12T20:57:35.813ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248422025-11-12T20:57:35.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248432025-11-12T20:57:35.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248442025-11-12T20:57:35.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248452025-11-12T20:57:35.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248462025-11-12T20:57:35.816ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248472025-11-12T20:57:35.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248482025-11-12T20:57:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248492025-11-12T20:57:35.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248502025-11-12T20:57:35.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248512025-11-12T20:57:35.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248522025-11-12T20:57:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248532025-11-12T20:57:35.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248542025-11-12T20:57:35.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248552025-11-12T20:57:35.822ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248562025-11-12T20:57:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248572025-11-12T20:57:35.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248582025-11-12T20:57:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248592025-11-12T20:57:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248602025-11-12T20:57:35.826ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
248612025-11-12T20:57:35.826ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248622025-11-12T20:57:35.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248632025-11-12T20:57:35.828ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248642025-11-12T20:57:35.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248652025-11-12T20:57:35.829ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
248662025-11-12T20:57:35.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248672025-11-12T20:57:35.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248682025-11-12T20:57:35.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248692025-11-12T20:57:35.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248702025-11-12T20:57:35.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248712025-11-12T20:57:35.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248722025-11-12T20:57:35.832ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248732025-11-12T20:57:35.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248742025-11-12T20:57:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248752025-11-12T20:57:35.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248762025-11-12T20:57:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248772025-11-12T20:57:35.834ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248782025-11-12T20:57:35.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248792025-11-12T20:57:35.836ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248802025-11-12T20:57:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248812025-11-12T20:57:35.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248822025-11-12T20:57:35.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248832025-11-12T20:57:35.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
248842025-11-12T20:57:35.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248852025-11-12T20:57:35.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248862025-11-12T20:57:35.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248872025-11-12T20:57:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, lookup_type: ById(22a04497-95f6-4ebc-818b-f7ecb7a5e4b7) }, key: 6e2011df-4294-4b70-ab38-3dc7dcbe2579, lookup_type: ById(6e2011df-4294-4b70-ab38-3dc7dcbe2579) }, key: a380daa6-a2af-48b0-9595-63b02b382af5, lookup_type: ById(a380daa6-a2af-48b0-9595-63b02b382af5) }, key: 4388a418-5a2b-4c91-8389-878cca6261d6, lookup_type: ById(4388a418-5a2b-4c91-8389-878cca6261d6) } result = Ok(())
248882025-11-12T20:57:35.845ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248892025-11-12T20:57:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248902025-11-12T20:57:35.846ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
248912025-11-12T20:57:35.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
248922025-11-12T20:57:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
248932025-11-12T20:57:35.847ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248942025-11-12T20:57:35.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248952025-11-12T20:57:35.848ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248962025-11-12T20:57:35.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248972025-11-12T20:57:35.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248982025-11-12T20:57:35.850ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248992025-11-12T20:57:35.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
249002025-11-12T20:57:35.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
249012025-11-12T20:57:35.852ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
249022025-11-12T20:57:35.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
249032025-11-12T20:57:35.853ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "50a85db0-c6b7-4c53-ac1a-959a4790c374", "content-length": "810", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
249042025-11-12T20:57:35.853ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(9271362) }} sled = sock
249052025-11-12T20:57:35.853ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
249062025-11-12T20:57:35.853ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
249072025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
249082025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
249092025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
249102025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, lookup_type: ById(22a04497-95f6-4ebc-818b-f7ecb7a5e4b7) }, key: 6e2011df-4294-4b70-ab38-3dc7dcbe2579, lookup_type: ById(6e2011df-4294-4b70-ab38-3dc7dcbe2579) }, key: a380daa6-a2af-48b0-9595-63b02b382af5, lookup_type: ById(a380daa6-a2af-48b0-9595-63b02b382af5) }, key: 4388a418-5a2b-4c91-8389-878cca6261d6, lookup_type: ById(4388a418-5a2b-4c91-8389-878cca6261d6) } result = Ok(())
249112025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249122025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249132025-11-12T20:57:35.855ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 578.330296ms iteration = 25 status = {}
249142025-11-12T20:57:35.857ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/b4864a61-939f-487c-a35b-cac96bdebfb2/state
249152025-11-12T20:57:35.859ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249162025-11-12T20:57:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249172025-11-12T20:57:35.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249182025-11-12T20:57:35.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249192025-11-12T20:57:35.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249202025-11-12T20:57:35.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249212025-11-12T20:57:35.866ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249222025-11-12T20:57:35.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249232025-11-12T20:57:35.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249242025-11-12T20:57:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249252025-11-12T20:57:35.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249262025-11-12T20:57:35.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249272025-11-12T20:57:35.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249282025-11-12T20:57:35.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249292025-11-12T20:57:35.876ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
249302025-11-12T20:57:35.878ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249312025-11-12T20:57:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, lookup_type: ById(22a04497-95f6-4ebc-818b-f7ecb7a5e4b7) }, key: 6e2011df-4294-4b70-ab38-3dc7dcbe2579, lookup_type: ById(6e2011df-4294-4b70-ab38-3dc7dcbe2579) }, key: a380daa6-a2af-48b0-9595-63b02b382af5, lookup_type: ById(a380daa6-a2af-48b0-9595-63b02b382af5) }, key: 0c6f89c6-27d9-41ab-8350-9ca674642ae8, lookup_type: ById(0c6f89c6-27d9-41ab-8350-9ca674642ae8) } result = Ok(())
249322025-11-12T20:57:35.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249332025-11-12T20:57:35.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249342025-11-12T20:57:35.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249352025-11-12T20:57:35.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249362025-11-12T20:57:35.892ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249372025-11-12T20:57:35.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249382025-11-12T20:57:35.894ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249392025-11-12T20:57:35.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249402025-11-12T20:57:35.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249412025-11-12T20:57:35.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249422025-11-12T20:57:35.898ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249432025-11-12T20:57:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249442025-11-12T20:57:35.899ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
249452025-11-12T20:57:35.901ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249462025-11-12T20:57:35.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249472025-11-12T20:57:35.903ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "f05b4d7a-4500-4363-a9c3-b519bbc24424", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
249482025-11-12T20:57:35.903ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
249492025-11-12T20:57:35.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249502025-11-12T20:57:35.905ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
249512025-11-12T20:57:35.905ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 615.457136ms iteration = 13 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
249522025-11-12T20:57:35.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, lookup_type: ById(22a04497-95f6-4ebc-818b-f7ecb7a5e4b7) }, key: 6e2011df-4294-4b70-ab38-3dc7dcbe2579, lookup_type: ById(6e2011df-4294-4b70-ab38-3dc7dcbe2579) }, key: a380daa6-a2af-48b0-9595-63b02b382af5, lookup_type: ById(a380daa6-a2af-48b0-9595-63b02b382af5) }, key: 0c6f89c6-27d9-41ab-8350-9ca674642ae8, lookup_type: ByName("default") } result = Ok(())
249532025-11-12T20:57:35.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249542025-11-12T20:57:35.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249552025-11-12T20:57:35.910ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249562025-11-12T20:57:35.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249572025-11-12T20:57:35.912ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249582025-11-12T20:57:35.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249592025-11-12T20:57:35.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249602025-11-12T20:57:35.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249612025-11-12T20:57:35.917ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249622025-11-12T20:57:35.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249632025-11-12T20:57:35.919ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249642025-11-12T20:57:35.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249652025-11-12T20:57:35.921ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249662025-11-12T20:57:35.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249672025-11-12T20:57:35.923ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249682025-11-12T20:57:35.926ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
249692025-11-12T20:57:35.926ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
249702025-11-12T20:57:35.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, lookup_type: ById(22a04497-95f6-4ebc-818b-f7ecb7a5e4b7) }, key: 6e2011df-4294-4b70-ab38-3dc7dcbe2579, lookup_type: ById(6e2011df-4294-4b70-ab38-3dc7dcbe2579) }, key: a380daa6-a2af-48b0-9595-63b02b382af5, lookup_type: ById(a380daa6-a2af-48b0-9595-63b02b382af5) }, key: a82ec15b-487e-4425-80d7-493e728f3e39, lookup_type: ByName("default") } result = Ok(())
249712025-11-12T20:57:35.929ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
249722025-11-12T20:57:35.937ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "98d51c69-076a-4a95-b0a2-dff385646079", "date": "Wed, 12 Nov 2025 20:57:35 GMT"} })
249732025-11-12T20:57:35.940ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 663.219133ms iteration = 37 status = {}
249742025-11-12T20:57:35.940ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 38 reason = Dependency
249752025-11-12T20:57:35.940ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
249762025-11-12T20:57:35.940ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249772025-11-12T20:57:35.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249782025-11-12T20:57:35.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249792025-11-12T20:57:35.945ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
249802025-11-12T20:57:35.945ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249812025-11-12T20:57:35.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249822025-11-12T20:57:35.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249832025-11-12T20:57:35.949ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
249842025-11-12T20:57:35.950ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249852025-11-12T20:57:35.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249862025-11-12T20:57:35.963ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1651, tv_nsec: 460779154 } })
249872025-11-12T20:57:36.013ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 739.481676ms iteration = 25 status = {}
249882025-11-12T20:57:36.025ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
249892025-11-12T20:57:36.026ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ff549426-fadf-4eb3-acb2-bc0e6a2bdfde", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249902025-11-12T20:57:36.026ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249912025-11-12T20:57:36.029ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a36c338e-5164-48c1-84cb-e7e24530d6f9", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249922025-11-12T20:57:36.029ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249932025-11-12T20:57:36.032ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ae877511-743c-4a20-b5ae-3bb14e139d27", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249942025-11-12T20:57:36.032ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249952025-11-12T20:57:36.034ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
249962025-11-12T20:57:36.034ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 760.861509ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
249972025-11-12T20:57:36.038ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
249982025-11-12T20:57:36.038ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3fcee358-e6d8-4556-819f-524024bb7eb2", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
249992025-11-12T20:57:36.038ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
250002025-11-12T20:57:36.041ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "04c1d187-04f7-413e-844f-256b1c678e71", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
250012025-11-12T20:57:36.041ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
250022025-11-12T20:57:36.043ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
250032025-11-12T20:57:36.043ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 770.295463ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
250042025-11-12T20:57:36.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250052025-11-12T20:57:36.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250062025-11-12T20:57:36.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250072025-11-12T20:57:36.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250082025-11-12T20:57:36.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250092025-11-12T20:57:36.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
250102025-11-12T20:57:36.068ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
250112025-11-12T20:57:36.072ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
250122025-11-12T20:57:36.072ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
250132025-11-12T20:57:36.074ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "be91a725-fd9a-4437-b1fa-245a34096c92", "content-length": "583", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
250142025-11-12T20:57:36.074ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250152025-11-12T20:57:36.074ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "bd6a09ff-4858-45db-8d23-c410b8afcfff", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
250162025-11-12T20:57:36.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250172025-11-12T20:57:36.076ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
250182025-11-12T20:57:36.076ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = Dns step id = Ensure
250192025-11-12T20:57:36.076ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
250202025-11-12T20:57:36.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250212025-11-12T20:57:36.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250222025-11-12T20:57:36.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250232025-11-12T20:57:36.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250242025-11-12T20:57:36.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250252025-11-12T20:57:36.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
250262025-11-12T20:57:36.091ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {494f8547-7581-4a15-8580-749d6dba22ca: {10.151.2.175: {a82ec15b-487e-4425-80d7-493e728f3e39}, 10.151.2.176: {a82ec15b-487e-4425-80d7-493e728f3e39}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
250272025-11-12T20:57:36.091ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
250282025-11-12T20:57:36.092ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "85f385c3-80e4-45f8-a4ca-2e7c7a76cd20", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
250292025-11-12T20:57:36.092ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250302025-11-12T20:57:36.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250312025-11-12T20:57:36.093ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250322025-11-12T20:57:36.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250332025-11-12T20:57:36.096ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250342025-11-12T20:57:36.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250352025-11-12T20:57:36.100ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250362025-11-12T20:57:36.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250372025-11-12T20:57:36.103ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250382025-11-12T20:57:36.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250392025-11-12T20:57:36.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250402025-11-12T20:57:36.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250412025-11-12T20:57:36.106ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
250422025-11-12T20:57:36.106ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250432025-11-12T20:57:36.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250442025-11-12T20:57:36.106ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
250452025-11-12T20:57:36.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
250462025-11-12T20:57:36.109ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
250472025-11-12T20:57:36.109ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
250482025-11-12T20:57:36.109ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
250492025-11-12T20:57:36.110ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "83ea8146-9733-4296-8b1f-613956916fbd", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
250502025-11-12T20:57:36.112ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 171.493949ms iteration = 38 status = {}
250512025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
250522025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
250532025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
250542025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
250552025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
250562025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
250572025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
250582025-11-12T20:57:36.119ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
250592025-11-12T20:57:36.120ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
250602025-11-12T20:57:36.120ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
250612025-11-12T20:57:36.120ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
250622025-11-12T20:57:36.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250632025-11-12T20:57:36.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250642025-11-12T20:57:36.122ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
250652025-11-12T20:57:36.122ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
250662025-11-12T20:57:36.122ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
250672025-11-12T20:57:36.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250682025-11-12T20:57:36.122ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
250692025-11-12T20:57:36.122ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
250702025-11-12T20:57:36.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250712025-11-12T20:57:36.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250722025-11-12T20:57:36.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
250732025-11-12T20:57:36.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250742025-11-12T20:57:36.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250752025-11-12T20:57:36.130ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
250762025-11-12T20:57:36.130ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
250772025-11-12T20:57:36.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250782025-11-12T20:57:36.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250792025-11-12T20:57:36.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250802025-11-12T20:57:36.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
250812025-11-12T20:57:36.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250822025-11-12T20:57:36.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250832025-11-12T20:57:36.137ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
250842025-11-12T20:57:36.137ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
250852025-11-12T20:57:36.137ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
250862025-11-12T20:57:36.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250872025-11-12T20:57:36.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250882025-11-12T20:57:36.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250892025-11-12T20:57:36.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250902025-11-12T20:57:36.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250912025-11-12T20:57:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
250922025-11-12T20:57:36.144ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
250932025-11-12T20:57:36.144ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
250942025-11-12T20:57:36.144ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
250952025-11-12T20:57:36.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
250962025-11-12T20:57:36.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
250972025-11-12T20:57:36.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250982025-11-12T20:57:36.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
250992025-11-12T20:57:36.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251002025-11-12T20:57:36.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251012025-11-12T20:57:36.150ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
251022025-11-12T20:57:36.150ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
251032025-11-12T20:57:36.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251042025-11-12T20:57:36.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251052025-11-12T20:57:36.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251062025-11-12T20:57:36.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
251072025-11-12T20:57:36.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251082025-11-12T20:57:36.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251092025-11-12T20:57:36.157ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
251102025-11-12T20:57:36.157ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
251112025-11-12T20:57:36.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251122025-11-12T20:57:36.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251132025-11-12T20:57:36.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251142025-11-12T20:57:36.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
251152025-11-12T20:57:36.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251162025-11-12T20:57:36.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251172025-11-12T20:57:36.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251182025-11-12T20:57:36.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251192025-11-12T20:57:36.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251202025-11-12T20:57:36.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251212025-11-12T20:57:36.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251222025-11-12T20:57:36.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
251232025-11-12T20:57:36.169ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
251242025-11-12T20:57:36.169ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
251252025-11-12T20:57:36.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = OmicronZones step id = Cleanup
251262025-11-12T20:57:36.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = OmicronZones step id = Cleanup
251272025-11-12T20:57:36.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = PhysicalDisks step id = Cleanup
251282025-11-12T20:57:36.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = Clickhouse step id = Ensure
251292025-11-12T20:57:36.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = Clickhouse step id = Ensure
251302025-11-12T20:57:36.200ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
251312025-11-12T20:57:36.204ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "7aebfa38-d502-413f-ba73-a5e90961cbe4", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
251322025-11-12T20:57:36.206ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = SupportBundles step id = Cleanup
251332025-11-12T20:57:36.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251342025-11-12T20:57:36.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251352025-11-12T20:57:36.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251362025-11-12T20:57:36.210ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
251372025-11-12T20:57:36.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251382025-11-12T20:57:36.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251392025-11-12T20:57:36.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251402025-11-12T20:57:36.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251412025-11-12T20:57:36.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251422025-11-12T20:57:36.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
251432025-11-12T20:57:36.222ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = OmicronZones step id = Cleanup
251442025-11-12T20:57:36.222ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
251452025-11-12T20:57:36.222ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
251462025-11-12T20:57:36.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251472025-11-12T20:57:36.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251482025-11-12T20:57:36.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251492025-11-12T20:57:36.225ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
251502025-11-12T20:57:36.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251512025-11-12T20:57:36.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251522025-11-12T20:57:36.244ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "00e61869-221d-4312-a878-cd9fa6862077", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
251532025-11-12T20:57:36.245ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
251542025-11-12T20:57:36.245ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
251552025-11-12T20:57:36.245ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7b8b9b6-278b-4b94-ab7f-040a3c53ae91", "content-length": "682", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
251562025-11-12T20:57:36.245ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
251572025-11-12T20:57:36.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
251582025-11-12T20:57:36.249ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
251592025-11-12T20:57:36.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
251602025-11-12T20:57:36.252ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
251612025-11-12T20:57:36.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
251622025-11-12T20:57:36.254ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
251632025-11-12T20:57:36.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
251642025-11-12T20:57:36.262ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 985.955695ms iteration = 25 status = {}
251652025-11-12T20:57:36.271ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
251662025-11-12T20:57:36.271ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
251672025-11-12T20:57:36.271ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = Cockroach step id = Ensure
251682025-11-12T20:57:36.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
251692025-11-12T20:57:36.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
251702025-11-12T20:57:36.280ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
251712025-11-12T20:57:36.280ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 05bbd10b-1e08-46cf-9bfb-73017bb91dad step component = MgsUpdates step id = Ensure
251722025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 38 reason = Dependency
251732025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Dependency
251742025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
251752025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
251762025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
251772025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.006727272s iteration = 14 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3481,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":91691863,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":91691813,"secs":0}},"event_index":1,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":91706123,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":27126292,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":27126242,"secs":0}},"event_index":2,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":118847356,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":21818364,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":21818314,"secs":0}},"event_index":3,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":140693671,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":301253476,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":301253436,"secs":0}},"event_index":4,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":441963088,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":360706534,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":360706474,"secs":0}},"event_index":5,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":802684152,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":93396367,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":93396327,"secs":0}},"event_index":6,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":896096059,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4320,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4290,"secs":0}},"event_index":7,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":896112020,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2310,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2270,"secs":0}},"event_index":8,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":896116590,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2880,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2850,"secs":0}},"event_index":9,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":896121580,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1640,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1600,"secs":0}},"event_index":10,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":896124900,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36678319,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36678259,"secs":0}},"event_index":11,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":932804709,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16233310,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16233270,"secs":0}},"event_index":12,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":949052910,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":48482465,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":48482415,"secs":0}},"event_index":13,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":997545685,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":8723616,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":8723566,"secs":0}},"event_index":14,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":6282171,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5780,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5730,"secs":0}},"event_index":15,"execution_id":"8330892e-1db9-4dd9-ad1c-eb65ee3101e1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":6296761,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
251782025-11-12T20:57:36.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
251792025-11-12T20:57:36.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251802025-11-12T20:57:36.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251812025-11-12T20:57:36.285ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
251822025-11-12T20:57:36.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
251832025-11-12T20:57:36.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
251842025-11-12T20:57:36.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
251852025-11-12T20:57:36.285ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
251862025-11-12T20:57:36.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
251872025-11-12T20:57:36.288ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
251882025-11-12T20:57:36.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
251892025-11-12T20:57:36.289ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
251902025-11-12T20:57:36.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
251912025-11-12T20:57:36.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
251922025-11-12T20:57:36.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
251932025-11-12T20:57:36.293ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
251942025-11-12T20:57:36.293ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 13.550351ms iteration = 38 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
251952025-11-12T20:57:36.398ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(7a3a620a-d640-4ac9-9394-c0f9ac495599, 0), (229ed602-3638-48e4-9414-366c12eb36a9, 0), (7bbb8b62-4d3b-4cf9-973b-db49da386151, 0), (737d712c-07bf-4cbc-9c46-02de218b0f65, 0)] collection_id = 7a3a620a-d640-4ac9-9394-c0f9ac495599
251962025-11-12T20:57:36.398ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
251972025-11-12T20:57:36.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
251982025-11-12T20:57:36.538ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 7a3a620a-d640-4ac9-9394-c0f9ac495599 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
251992025-11-12T20:57:36.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
252002025-11-12T20:57:36.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
252012025-11-12T20:57:36.614ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(7bbb8b62-4d3b-4cf9-973b-db49da386151, 0), (737d712c-07bf-4cbc-9c46-02de218b0f65, 0), (98e4220e-c01e-4ccf-adc3-06f59cdfaa04, 0)]
252022025-11-12T20:57:36.615ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 112588922 } })
252032025-11-12T20:57:36.646ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 144391045 } })
252042025-11-12T20:57:36.647ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
252052025-11-12T20:57:36.647ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
252062025-11-12T20:57:36.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
252072025-11-12T20:57:36.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "80f857df-4a2d-4351-8995-c962b1f0f3c4", "content-length": "476", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252082025-11-12T20:57:36.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
252092025-11-12T20:57:36.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5da1377b-e30d-4495-8f3b-f519ce3399f7", "content-length": "734", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252102025-11-12T20:57:36.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
252112025-11-12T20:57:36.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f953a71-3728-458b-b8b1-99d40508253b", "content-length": "111", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252122025-11-12T20:57:36.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
252132025-11-12T20:57:36.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3249d437-5ddc-4b5b-b3a3-b502ae874213", "content-length": "111", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252142025-11-12T20:57:36.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
252152025-11-12T20:57:36.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fae2f73c-a382-42c8-85e3-8330bbde33f9", "content-length": "163", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252162025-11-12T20:57:36.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
252172025-11-12T20:57:36.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7388e70e-67c0-4b15-9eb8-242655abd7b5", "content-length": "163", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252182025-11-12T20:57:36.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
252192025-11-12T20:57:36.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ed62602-1783-4b21-91e6-62821de02fc3", "content-length": "166", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252202025-11-12T20:57:36.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
252212025-11-12T20:57:36.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abe5195e-4426-4756-9201-aa9b567a3820", "content-length": "166", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252222025-11-12T20:57:36.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
252232025-11-12T20:57:36.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b729d32-ecb8-40b2-9275-42013abd926d", "content-length": "702", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252242025-11-12T20:57:36.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
252252025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c10ea16-3bac-44a2-b0a3-ef2d27d6f823", "content-length": "727", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252262025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
252272025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa413f70-0541-4abb-b772-00840c806211", "content-length": "729", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252282025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
252292025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a8a5f81-8fe9-4f52-be4a-8111834ab3fe", "content-length": "728", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252302025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
252312025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c9c273d-78e5-4f2b-b974-d25b00c1a4ba", "content-length": "722", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252322025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
252332025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bbaa84a-38c7-43ef-a19d-a8c779a61d6c", "content-length": "10", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252342025-11-12T20:57:36.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
252352025-11-12T20:57:36.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "219f1a8d-529b-466d-ac12-8cabfb45e0aa", "content-length": "144", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252362025-11-12T20:57:36.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
252372025-11-12T20:57:36.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c157883b-671c-4655-86c5-f7db8605a22e", "content-length": "147", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252382025-11-12T20:57:36.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
252392025-11-12T20:57:36.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "382903e4-04a1-4a04-9e48-0668db141a47", "content-length": "109", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252402025-11-12T20:57:36.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
252412025-11-12T20:57:36.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc4c988-6669-4964-a7e3-8c3eb7575582", "content-length": "109", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252422025-11-12T20:57:36.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
252432025-11-12T20:57:36.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a07d561-45eb-4887-95cd-1d106695d5f5", "content-length": "163", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252442025-11-12T20:57:36.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
252452025-11-12T20:57:36.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "be9a42b5-b668-44c2-ae60-95e22798d340", "content-length": "163", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252462025-11-12T20:57:36.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
252472025-11-12T20:57:36.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6851041-0e91-4688-9b56-0009cbcd81c6", "content-length": "166", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252482025-11-12T20:57:36.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
252492025-11-12T20:57:36.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb4ca3b-5c48-42c1-9b5d-ad037bc5249f", "content-length": "166", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252502025-11-12T20:57:36.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
252512025-11-12T20:57:36.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb2eb948-67a5-4d09-acd3-fcb6a367e055", "content-length": "702", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252522025-11-12T20:57:36.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
252532025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2fcc0aa-7de4-43b9-89bf-84abd82647b9", "content-length": "727", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252542025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
252552025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7dd502b-56c2-4a5f-8299-379592cd7241", "content-length": "729", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252562025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
252572025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62492e4e-2886-4b5c-a145-eb0d4a60227a", "content-length": "728", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252582025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
252592025-11-12T20:57:36.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
252602025-11-12T20:57:36.657ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
252612025-11-12T20:57:36.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
252622025-11-12T20:57:36.659ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
252632025-11-12T20:57:36.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
252642025-11-12T20:57:36.661ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
252652025-11-12T20:57:36.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
252662025-11-12T20:57:36.663ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
252672025-11-12T20:57:36.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
252682025-11-12T20:57:36.698ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
252692025-11-12T20:57:36.698ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
252702025-11-12T20:57:36.838ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe299407-0d77-49a5-bfb1-8f431233245b", "content-length": "95396", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252712025-11-12T20:57:36.840ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
252722025-11-12T20:57:36.840ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
252732025-11-12T20:57:36.840ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
252742025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
252752025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
252762025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
252772025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
252782025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
252792025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "907149ce-24cf-4c5e-be3c-802cc2571483", "content-length": "64", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252802025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8805f80-1632-4112-8928-7550a3ad1bb6", "content-length": "64", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252812025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
252822025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
252832025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4339cbc6-43ed-4611-a584-a489bee42271", "content-length": "64", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252842025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0ae5f2b-7ca5-40d5-8e35-f623bdeff47e", "content-length": "64", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252852025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
252862025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
252872025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "773449ba-dbd5-43da-9dc6-ee005282946d", "content-length": "64", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252882025-11-12T20:57:36.840ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
252892025-11-12T20:57:36.877ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cde7327e-19ae-4b7d-9281-888418762c67", "content-length": "409016", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252902025-11-12T20:57:36.881ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a2860ab-2303-4624-8e26-8090092d759c", "content-length": "415162", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252912025-11-12T20:57:36.883ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbab16b0-a8a6-47b2-89f9-955e7b05732a", "content-length": "413749", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252922025-11-12T20:57:36.885ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "61eb8174-1aa1-495b-bdef-95a8a2d4ea74", "content-length": "418936", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252932025-11-12T20:57:36.887ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed227fa0-99b8-40e3-9920-d692c4a85683", "content-length": "437303", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
252942025-11-12T20:57:36.890ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
252952025-11-12T20:57:36.890ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
252962025-11-12T20:57:36.890ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
252972025-11-12T20:57:36.890ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
252982025-11-12T20:57:36.890ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
252992025-11-12T20:57:36.921ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
253002025-11-12T20:57:36.921ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
253012025-11-12T20:57:36.924ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "91f5b7ea-4c65-4691-b2c9-2d441b5f5eeb", "content-length": "125", "date": "Wed, 12 Nov 2025 20:57:36 GMT"} })
253022025-11-12T20:57:36.925ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
253032025-11-12T20:57:37.019ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
253042025-11-12T20:57:37.019ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
253052025-11-12T20:57:37.020ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "26702cc8-ea78-4e72-b690-52c3c4930a84", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:57:37 GMT"} })
253062025-11-12T20:57:37.020ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
253072025-11-12T20:57:37.022ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
253082025-11-12T20:57:37.022ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
253092025-11-12T20:57:37.023ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1db6ebe7-659f-4a5a-a4c1-6cc28917cc7c", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:57:37 GMT"} })
253102025-11-12T20:57:37.023ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
253112025-11-12T20:57:37.024ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
253122025-11-12T20:57:37.024ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
253132025-11-12T20:57:37.025ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfcdf1b6-476a-4c5c-a758-a4a79804e512", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:57:37 GMT"} })
253142025-11-12T20:57:37.025ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
253152025-11-12T20:57:37.027ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
253162025-11-12T20:57:37.027ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
253172025-11-12T20:57:37.029ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
253182025-11-12T20:57:37.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
253192025-11-12T20:57:37.031ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253202025-11-12T20:57:37.034ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
253212025-11-12T20:57:37.034ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
253222025-11-12T20:57:37.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
253232025-11-12T20:57:37.229ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = df7e6024-e0ad-4d74-ae9f-796a61ac3daf file = nexus/db-queries/src/db/datastore/inventory.rs:1614
253242025-11-12T20:57:37.229ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = df7e6024-e0ad-4d74-ae9f-796a61ac3daf time_started = 2025-11-12 20:57:36.647597 UTC
253252025-11-12T20:57:37.229ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 64 reason = Dependency
253262025-11-12T20:57:37.229ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
253272025-11-12T20:57:37.229ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 949.706896ms iteration = 15 status = {"collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","time_done":"2025-11-12 20:57:37.027311 UTC","time_started":"2025-11-12 20:57:36.647597 UTC"}
253282025-11-12T20:57:37.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
253292025-11-12T20:57:37.231ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253302025-11-12T20:57:37.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
253312025-11-12T20:57:37.235ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
253322025-11-12T20:57:37.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
253332025-11-12T20:57:37.239ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
253342025-11-12T20:57:37.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
253352025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 62.889247ms iteration = 64 status = {"Loaded":{"collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","time_loaded":"2025-11-12T20:57:37.229863754Z","time_started":"2025-11-12T20:57:36.647597Z"}}
253362025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 41 reason = Dependency
253372025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 31 reason = Dependency
253382025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 19 reason = Dependency
253392025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253402025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
253412025-11-12T20:57:37.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253422025-11-12T20:57:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253432025-11-12T20:57:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
253442025-11-12T20:57:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253452025-11-12T20:57:37.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253462025-11-12T20:57:37.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253472025-11-12T20:57:37.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253482025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
253492025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253502025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
253512025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253522025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253532025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
253542025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
253552025-11-12T20:57:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253562025-11-12T20:57:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
253572025-11-12T20:57:37.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253582025-11-12T20:57:37.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253592025-11-12T20:57:37.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
253602025-11-12T20:57:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
253612025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253622025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
253632025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253642025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253652025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
253662025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253672025-11-12T20:57:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253682025-11-12T20:57:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
253692025-11-12T20:57:37.305ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
253702025-11-12T20:57:37.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253712025-11-12T20:57:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
253722025-11-12T20:57:37.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253732025-11-12T20:57:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253742025-11-12T20:57:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253752025-11-12T20:57:37.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253762025-11-12T20:57:37.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253772025-11-12T20:57:37.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
253782025-11-12T20:57:37.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253792025-11-12T20:57:37.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253802025-11-12T20:57:37.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
253812025-11-12T20:57:37.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253822025-11-12T20:57:37.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253832025-11-12T20:57:37.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253842025-11-12T20:57:37.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253852025-11-12T20:57:37.313ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.115507ms iteration = 41 status = {"physical_disks_added":0}
253862025-11-12T20:57:37.314ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
253872025-11-12T20:57:37.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253882025-11-12T20:57:37.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253892025-11-12T20:57:37.315ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
253902025-11-12T20:57:37.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253912025-11-12T20:57:37.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253922025-11-12T20:57:37.317ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
253932025-11-12T20:57:37.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
253942025-11-12T20:57:37.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
253952025-11-12T20:57:37.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
253962025-11-12T20:57:37.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
253972025-11-12T20:57:37.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
253982025-11-12T20:57:37.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
253992025-11-12T20:57:37.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
254002025-11-12T20:57:37.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254012025-11-12T20:57:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
254022025-11-12T20:57:37.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254032025-11-12T20:57:37.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254042025-11-12T20:57:37.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254052025-11-12T20:57:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
254062025-11-12T20:57:37.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254072025-11-12T20:57:37.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254082025-11-12T20:57:37.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
254092025-11-12T20:57:37.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
254102025-11-12T20:57:37.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
254112025-11-12T20:57:37.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254122025-11-12T20:57:37.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254132025-11-12T20:57:37.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254142025-11-12T20:57:37.329ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
254152025-11-12T20:57:37.329ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 36.976966ms iteration = 19 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
254162025-11-12T20:57:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
254172025-11-12T20:57:37.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254182025-11-12T20:57:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254192025-11-12T20:57:37.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254202025-11-12T20:57:37.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
254212025-11-12T20:57:37.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254222025-11-12T20:57:37.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254232025-11-12T20:57:37.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254242025-11-12T20:57:37.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254252025-11-12T20:57:37.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254262025-11-12T20:57:37.344ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
254272025-11-12T20:57:37.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254282025-11-12T20:57:37.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254292025-11-12T20:57:37.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254302025-11-12T20:57:37.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254312025-11-12T20:57:37.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254322025-11-12T20:57:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
254332025-11-12T20:57:37.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254342025-11-12T20:57:37.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254352025-11-12T20:57:37.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254362025-11-12T20:57:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254372025-11-12T20:57:37.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254382025-11-12T20:57:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
254392025-11-12T20:57:37.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254402025-11-12T20:57:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254412025-11-12T20:57:37.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254422025-11-12T20:57:37.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254432025-11-12T20:57:37.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254442025-11-12T20:57:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254452025-11-12T20:57:37.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254462025-11-12T20:57:37.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
254472025-11-12T20:57:37.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254482025-11-12T20:57:37.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254492025-11-12T20:57:37.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254502025-11-12T20:57:37.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
254512025-11-12T20:57:37.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254522025-11-12T20:57:37.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254532025-11-12T20:57:37.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254542025-11-12T20:57:37.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254552025-11-12T20:57:37.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254562025-11-12T20:57:37.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254572025-11-12T20:57:37.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254582025-11-12T20:57:37.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
254592025-11-12T20:57:37.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254602025-11-12T20:57:37.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254612025-11-12T20:57:37.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254622025-11-12T20:57:37.393ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
254632025-11-12T20:57:37.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254642025-11-12T20:57:37.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254652025-11-12T20:57:37.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254662025-11-12T20:57:37.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254672025-11-12T20:57:37.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254682025-11-12T20:57:37.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254692025-11-12T20:57:37.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254702025-11-12T20:57:37.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
254712025-11-12T20:57:37.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254722025-11-12T20:57:37.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254732025-11-12T20:57:37.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254742025-11-12T20:57:37.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254752025-11-12T20:57:37.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254762025-11-12T20:57:37.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
254772025-11-12T20:57:37.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254782025-11-12T20:57:37.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254792025-11-12T20:57:37.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254802025-11-12T20:57:37.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254812025-11-12T20:57:37.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254822025-11-12T20:57:37.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
254832025-11-12T20:57:37.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254842025-11-12T20:57:37.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254852025-11-12T20:57:37.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254862025-11-12T20:57:37.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254872025-11-12T20:57:37.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254882025-11-12T20:57:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
254892025-11-12T20:57:37.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254902025-11-12T20:57:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254912025-11-12T20:57:37.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254922025-11-12T20:57:37.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254932025-11-12T20:57:37.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
254942025-11-12T20:57:37.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
254952025-11-12T20:57:37.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254962025-11-12T20:57:37.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254972025-11-12T20:57:37.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
254982025-11-12T20:57:37.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
254992025-11-12T20:57:37.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255002025-11-12T20:57:37.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
255012025-11-12T20:57:37.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
255022025-11-12T20:57:37.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
255032025-11-12T20:57:37.472ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
255042025-11-12T20:57:37.472ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
255052025-11-12T20:57:37.472ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = c1f990aa-7696-4384-97b3-52a3314e8cea parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
255062025-11-12T20:57:37.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
255072025-11-12T20:57:37.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
255082025-11-12T20:57:37.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255092025-11-12T20:57:37.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
255102025-11-12T20:57:37.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
255112025-11-12T20:57:37.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
255122025-11-12T20:57:37.479ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
255132025-11-12T20:57:37.479ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
255142025-11-12T20:57:37.480ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 187.289136ms iteration = 31 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
255152025-11-12T20:57:37.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 363 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 17845d0a-d3c7-480a-aa1e-f8c6bc9e1ea6 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
255162025-11-12T20:57:38.992ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/b4864a61-939f-487c-a35b-cac96bdebfb2/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "80b09f3a-aa85-423d-b0d7-32b670dc430d", "content-length": "131", "date": "Wed, 12 Nov 2025 20:57:38 GMT"} })
255172025-11-12T20:57:38.997ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-11-12T20:57:30.534017797Z }, migration_in: None, migration_out: None } vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2
255182025-11-12T20:57:39.003ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2152 instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b4864a61-939f-487c-a35b-cac96bdebfb2 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-11-12T20:57:30.534017797Z }
255192025-11-12T20:57:39.003ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 roles = RoleSet { roles: {} } sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2
255202025-11-12T20:57:39.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 resource = Database result = Ok(()) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2
255212025-11-12T20:57:39.032ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
255222025-11-12T20:57:39.034ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 3.757550888s iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"starting":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
255232025-11-12T20:57:40.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 146 reason = Timeout
255242025-11-12T20:57:40.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
255252025-11-12T20:57:40.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
255262025-11-12T20:57:40.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255272025-11-12T20:57:40.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
255282025-11-12T20:57:40.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
255292025-11-12T20:57:40.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
255302025-11-12T20:57:40.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
255312025-11-12T20:57:40.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.984967ms iteration = 146 status = {"config_updated":false}
255322025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
255332025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
255342025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
255352025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 147 reason = Timeout
255362025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
255372025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
255382025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
255392025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
255402025-11-12T20:57:45.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
255412025-11-12T20:57:45.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255422025-11-12T20:57:45.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
255432025-11-12T20:57:45.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255442025-11-12T20:57:45.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
255452025-11-12T20:57:45.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.930611ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
255462025-11-12T20:57:45.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
255472025-11-12T20:57:45.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
255482025-11-12T20:57:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
255492025-11-12T20:57:45.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
255502025-11-12T20:57:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
255512025-11-12T20:57:45.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
255522025-11-12T20:57:45.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
255532025-11-12T20:57:45.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.119744ms iteration = 147 status = {"config_updated":false}
255542025-11-12T20:57:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
255552025-11-12T20:57:45.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
255562025-11-12T20:57:45.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255572025-11-12T20:57:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
255582025-11-12T20:57:45.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
255592025-11-12T20:57:45.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
255602025-11-12T20:57:45.302ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
255612025-11-12T20:57:45.302ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.50846ms iteration = 74 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
255622025-11-12T20:57:47.913ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 4e36b434-3903-4467-8e95-568d180f8cc0 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
255632025-11-12T20:57:47.913ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 4e36b434-3903-4467-8e95-568d180f8cc0 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
255642025-11-12T20:57:47.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 4e36b434-3903-4467-8e95-568d180f8cc0 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
255652025-11-12T20:57:47.916ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3319 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 4e36b434-3903-4467-8e95-568d180f8cc0 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
255662025-11-12T20:57:47.917ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 35fe6ee3-6f01-48bc-aebb-0301a496fb54 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
255672025-11-12T20:57:47.917ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 35fe6ee3-6f01-48bc-aebb-0301a496fb54 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
255682025-11-12T20:57:47.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 35fe6ee3-6f01-48bc-aebb-0301a496fb54 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
255692025-11-12T20:57:47.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2537 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:50720 req_id = 35fe6ee3-6f01-48bc-aebb-0301a496fb54 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
255702025-11-12T20:57:47.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 442 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 7d1ce8d4-2367-4804-a047-fa2add521224 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
255712025-11-12T20:57:50.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 65 reason = Timeout
255722025-11-12T20:57:50.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
255732025-11-12T20:57:50.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
255742025-11-12T20:57:50.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 148 reason = Timeout
255752025-11-12T20:57:50.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
255762025-11-12T20:57:50.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
255772025-11-12T20:57:50.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255782025-11-12T20:57:50.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255792025-11-12T20:57:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
255802025-11-12T20:57:50.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
255812025-11-12T20:57:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
255822025-11-12T20:57:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
255832025-11-12T20:57:50.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
255842025-11-12T20:57:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
255852025-11-12T20:57:50.282ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df7e6024-e0ad-4d74-ae9f-796a61ac3daf original_time_started = 2025-11-12 20:57:36.647597 UTC
255862025-11-12T20:57:50.282ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.091304ms iteration = 65 status = {"Loaded":{"collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","time_loaded":"2025-11-12T20:57:50.275704429Z","time_started":"2025-11-12T20:57:36.647597Z"}}
255872025-11-12T20:57:50.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
255882025-11-12T20:57:50.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 50 reason = Timeout
255892025-11-12T20:57:50.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.637412ms iteration = 148 status = {"config_updated":false}
255902025-11-12T20:57:50.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
255912025-11-12T20:57:50.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
255922025-11-12T20:57:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
255932025-11-12T20:57:50.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
255942025-11-12T20:57:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
255952025-11-12T20:57:50.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
255962025-11-12T20:57:50.288ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 4.987439ms iteration = 50 status = "NoSitrep"
255972025-11-12T20:57:50.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 50 reason = Timeout
255982025-11-12T20:57:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
255992025-11-12T20:57:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
256002025-11-12T20:57:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
256012025-11-12T20:57:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
256022025-11-12T20:57:50.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
256032025-11-12T20:57:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
256042025-11-12T20:57:50.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
256052025-11-12T20:57:50.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
256062025-11-12T20:57:50.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
256072025-11-12T20:57:50.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
256082025-11-12T20:57:50.298ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
256092025-11-12T20:57:50.299ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
256102025-11-12T20:57:50.300ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
256112025-11-12T20:57:50.300ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.196514ms iteration = 50 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
256122025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
256132025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
256142025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
256152025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 149 reason = Timeout
256162025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
256172025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
256182025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
256192025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
256202025-11-12T20:57:55.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
256212025-11-12T20:57:55.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
256222025-11-12T20:57:55.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
256232025-11-12T20:57:55.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
256242025-11-12T20:57:55.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
256252025-11-12T20:57:55.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.590713ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
256262025-11-12T20:57:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
256272025-11-12T20:57:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
256282025-11-12T20:57:55.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
256292025-11-12T20:57:55.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
256302025-11-12T20:57:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
256312025-11-12T20:57:55.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
256322025-11-12T20:57:55.282ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
256332025-11-12T20:57:55.282ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.295103ms iteration = 149 status = {"config_updated":false}
256342025-11-12T20:57:55.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
256352025-11-12T20:57:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
256362025-11-12T20:57:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
256372025-11-12T20:57:55.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
256382025-11-12T20:57:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
256392025-11-12T20:57:55.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
256402025-11-12T20:57:55.300ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
256412025-11-12T20:57:55.300ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.784106ms iteration = 75 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
256422025-11-12T20:57:57.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 509 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 90cb39ae-32bd-4b53-b1c8-358ddbc74177 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
256432025-11-12T20:58:00.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 150 reason = Timeout
256442025-11-12T20:58:00.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
256452025-11-12T20:58:00.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
256462025-11-12T20:58:00.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
256472025-11-12T20:58:00.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
256482025-11-12T20:58:00.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
256492025-11-12T20:58:00.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
256502025-11-12T20:58:00.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
256512025-11-12T20:58:00.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.049632ms iteration = 150 status = {"config_updated":false}
256522025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
256532025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
256542025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
256552025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
256562025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
256572025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
256582025-11-12T20:58:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
256592025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 66 reason = Timeout
256602025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
256612025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
256622025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 39 reason = Timeout
256632025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
256642025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
256652025-11-12T20:58:05.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
256662025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
256672025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
256682025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
256692025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
256702025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
256712025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
256722025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 42 reason = Timeout
256732025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
256742025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
256752025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
256762025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
256772025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
256782025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
256792025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
256802025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
256812025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
256822025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
256832025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
256842025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 151 reason = Timeout
256852025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
256862025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
256872025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
256882025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
256892025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
256902025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
256912025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
256922025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
256932025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
256942025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 39 reason = Timeout
256952025-11-12T20:58:05.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
256962025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
256972025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
256982025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
256992025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
257002025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
257012025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
257022025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
257032025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
257042025-11-12T20:58:05.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 775173069 } })
257052025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
257062025-11-12T20:58:05.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 775263181 } })
257072025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
257082025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
257092025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
257102025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
257112025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
257122025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
257132025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
257142025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
257152025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
257162025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
257172025-11-12T20:58:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
257182025-11-12T20:58:05.278ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 776088002 } })
257192025-11-12T20:58:05.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
257202025-11-12T20:58:05.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257212025-11-12T20:58:05.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257222025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
257232025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257242025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257252025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257262025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
257272025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
257282025-11-12T20:58:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
257292025-11-12T20:58:05.281ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.089863ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
257302025-11-12T20:58:05.281ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257312025-11-12T20:58:05.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257322025-11-12T20:58:05.282ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257332025-11-12T20:58:05.283ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.920613ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
257342025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257352025-11-12T20:58:05.284ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
257362025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 51 reason = Timeout
257372025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
257382025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
257392025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
257402025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
257412025-11-12T20:58:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
257422025-11-12T20:58:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
257432025-11-12T20:58:05.285ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
257442025-11-12T20:58:05.286ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
257452025-11-12T20:58:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
257462025-11-12T20:58:05.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257472025-11-12T20:58:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
257482025-11-12T20:58:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
257492025-11-12T20:58:05.287ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
257502025-11-12T20:58:05.287ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
257512025-11-12T20:58:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
257522025-11-12T20:58:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
257532025-11-12T20:58:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
257542025-11-12T20:58:05.288ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.940633ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
257552025-11-12T20:58:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
257562025-11-12T20:58:05.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.152198ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
257572025-11-12T20:58:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
257582025-11-12T20:58:05.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
257592025-11-12T20:58:05.288ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1680, tv_nsec: 786462831 } })
257602025-11-12T20:58:05.288ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
257612025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
257622025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
257632025-11-12T20:58:05.289ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
257642025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
257652025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
257662025-11-12T20:58:05.289ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
257672025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
257682025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
257692025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
257702025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
257712025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
257722025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 51 reason = Timeout
257732025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
257742025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 12.514575ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
257752025-11-12T20:58:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
257762025-11-12T20:58:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
257772025-11-12T20:58:05.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
257782025-11-12T20:58:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
257792025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
257802025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257812025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
257822025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
257832025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
257842025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
257852025-11-12T20:58:05.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
257862025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
257872025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
257882025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
257892025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
257902025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
257912025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
257922025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
257932025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257942025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
257952025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
257962025-11-12T20:58:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
257972025-11-12T20:58:05.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257982025-11-12T20:58:05.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
257992025-11-12T20:58:05.294ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df7e6024-e0ad-4d74-ae9f-796a61ac3daf original_time_started = 2025-11-12 20:57:36.647597 UTC
258002025-11-12T20:58:05.294ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 17.993895ms iteration = 66 status = {"Loaded":{"collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","time_loaded":"2025-11-12T20:58:05.276470142Z","time_started":"2025-11-12T20:57:36.647597Z"}}
258012025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 11.065756ms iteration = 51 status = "NoSitrep"
258022025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
258032025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 17.945864ms iteration = 151 status = {"config_updated":false}
258042025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
258052025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
258062025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
258072025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 18.963991ms iteration = 26 status = 0
258082025-11-12T20:58:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
258092025-11-12T20:58:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
258102025-11-12T20:58:05.299ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
258112025-11-12T20:58:05.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
258122025-11-12T20:58:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
258132025-11-12T20:58:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
258142025-11-12T20:58:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
258152025-11-12T20:58:05.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
258162025-11-12T20:58:05.300ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
258172025-11-12T20:58:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
258182025-11-12T20:58:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
258192025-11-12T20:58:05.302ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
258202025-11-12T20:58:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
258212025-11-12T20:58:05.305ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
258222025-11-12T20:58:05.305ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 16.006394ms iteration = 51 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
258232025-11-12T20:58:05.315ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 38.811794ms iteration = 42 status = {"physical_disks_added":0}
258242025-11-12T20:58:05.324ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
258252025-11-12T20:58:05.324ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.463338ms iteration = 76 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
258262025-11-12T20:58:05.329ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
258272025-11-12T20:58:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
258282025-11-12T20:58:05.329ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 52.780717ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
258292025-11-12T20:58:05.331ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
258302025-11-12T20:58:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
258312025-11-12T20:58:05.333ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.727829ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
258322025-11-12T20:58:05.337ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
258332025-11-12T20:58:05.337ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
258342025-11-12T20:58:05.337ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 60.831824ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
258352025-11-12T20:58:05.350ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
258362025-11-12T20:58:05.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
258372025-11-12T20:58:05.351ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
258382025-11-12T20:58:05.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
258392025-11-12T20:58:05.352ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
258402025-11-12T20:58:05.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
258412025-11-12T20:58:05.359ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
258422025-11-12T20:58:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
258432025-11-12T20:58:05.364ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
258442025-11-12T20:58:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
258452025-11-12T20:58:05.367ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
258462025-11-12T20:58:05.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
258472025-11-12T20:58:05.369ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
258482025-11-12T20:58:05.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
258492025-11-12T20:58:05.371ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
258502025-11-12T20:58:05.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
258512025-11-12T20:58:05.374ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
258522025-11-12T20:58:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
258532025-11-12T20:58:05.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 108.94525ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
258542025-11-12T20:58:05.416ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
258552025-11-12T20:58:05.416ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 139.621143ms iteration = 39 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
258562025-11-12T20:58:05.423ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
258572025-11-12T20:58:05.425ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 150.910236ms iteration = 26 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
258582025-11-12T20:58:05.432ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
258592025-11-12T20:58:05.432ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
258602025-11-12T20:58:05.432ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a137e66-fc05-4cfe-82bc-35378e802ac7", "content-length": "26", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258612025-11-12T20:58:05.432ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
258622025-11-12T20:58:05.434ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
258632025-11-12T20:58:05.445ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
258642025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bedc25f-c908-49d3-944d-966a653f793e", "content-length": "78", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258652025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
258662025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
258672025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
258682025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
258692025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
258702025-11-12T20:58:05.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
258712025-11-12T20:58:05.446ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
258722025-11-12T20:58:05.446ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
258732025-11-12T20:58:05.446ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "92cf670e-45e3-44e7-a9be-02ed299bfd70", "content-length": "26", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258742025-11-12T20:58:05.447ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
258752025-11-12T20:58:05.448ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
258762025-11-12T20:58:05.448ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
258772025-11-12T20:58:05.448ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
258782025-11-12T20:58:05.448ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
258792025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
258802025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
258812025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
258822025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
258832025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
258842025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
258852025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
258862025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
258872025-11-12T20:58:05.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
258882025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
258892025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
258902025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
258912025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
258922025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
258932025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
258942025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "55946ff4-9141-476a-8414-7d94c316ea99", "content-length": "81", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258952025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c28afd4-228f-4515-8a0a-ec4884b0342f", "content-length": "81", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258962025-11-12T20:58:05.454ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7fe73da-a264-467f-a211-3af9438c1867", "content-length": "81", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258972025-11-12T20:58:05.455ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/b4864a61-939f-487c-a35b-cac96bdebfb2/state
258982025-11-12T20:58:05.455ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/b4864a61-939f-487c-a35b-cac96bdebfb2/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "88333973-f551-4940-928d-a33f46579d33", "content-length": "130", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
258992025-11-12T20:58:05.455ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-11-12T20:57:39.084560909Z }, migration_in: None, migration_out: None } vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2
259002025-11-12T20:58:05.455ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2152 instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b4864a61-939f-487c-a35b-cac96bdebfb2 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-11-12T20:57:39.084560909Z }
259012025-11-12T20:58:05.455ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 roles = RoleSet { roles: {} } sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2
259022025-11-12T20:58:05.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher instance_id = 9dbebd19-ea8f-407c-88dc-4abedbce5e12 resource = Database result = Ok(()) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 vmm_id = b4864a61-939f-487c-a35b-cac96bdebfb2
259032025-11-12T20:58:05.456ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 179.31733ms iteration = 26 status = {"disabled":false,"errors":[],"sps":[]}
259042025-11-12T20:58:05.460ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
259052025-11-12T20:58:05.463ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 185.827359ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
259062025-11-12T20:58:05.512ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
259072025-11-12T20:58:05.513ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcfad6d7-73b3-497e-9b36-164f79b40009", "content-length": "810", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
259082025-11-12T20:58:05.513ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(9271362) }} sled = sock
259092025-11-12T20:58:05.513ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
259102025-11-12T20:58:05.513ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
259112025-11-12T20:58:05.515ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 238.156823ms iteration = 26 status = {}
259122025-11-12T20:58:05.530ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
259132025-11-12T20:58:05.530ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
259142025-11-12T20:58:05.530ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3efda348-5039-4174-a4f3-9982fdaced24", "content-length": "583", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
259152025-11-12T20:58:05.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259162025-11-12T20:58:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259172025-11-12T20:58:05.532ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 258.189061ms iteration = 26 status = {}
259182025-11-12T20:58:05.534ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1681, tv_nsec: 32078477 } })
259192025-11-12T20:58:05.543ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {494f8547-7581-4a15-8580-749d6dba22ca: {10.151.2.176: {a82ec15b-487e-4425-80d7-493e728f3e39}, 10.151.2.175: {a82ec15b-487e-4425-80d7-493e728f3e39}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
259202025-11-12T20:58:05.543ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
259212025-11-12T20:58:05.544ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "53c15e37-009e-4ade-b4d8-590699ede948", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
259222025-11-12T20:58:05.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259232025-11-12T20:58:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259242025-11-12T20:58:05.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259252025-11-12T20:58:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259262025-11-12T20:58:05.548ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259272025-11-12T20:58:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259282025-11-12T20:58:05.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259292025-11-12T20:58:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259302025-11-12T20:58:05.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259312025-11-12T20:58:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259322025-11-12T20:58:05.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
259332025-11-12T20:58:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
259342025-11-12T20:58:05.559ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
259352025-11-12T20:58:05.559ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
259362025-11-12T20:58:05.559ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
259372025-11-12T20:58:05.560ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "23d6015c-0aa6-450a-b1a2-597158d3ac93", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
259382025-11-12T20:58:05.562ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 285.035636ms iteration = 39 status = {}
259392025-11-12T20:58:05.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259402025-11-12T20:58:05.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259412025-11-12T20:58:05.599ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
259422025-11-12T20:58:05.599ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259432025-11-12T20:58:05.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259442025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
259452025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
259462025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
259472025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
259482025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
259492025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
259502025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
259512025-11-12T20:58:05.612ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
259522025-11-12T20:58:05.613ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
259532025-11-12T20:58:05.613ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
259542025-11-12T20:58:05.613ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
259552025-11-12T20:58:05.613ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259562025-11-12T20:58:05.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259572025-11-12T20:58:05.616ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
259582025-11-12T20:58:05.616ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
259592025-11-12T20:58:05.723ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "da2f93bc-72f8-409b-8a94-3ce6ffb850a7", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
259602025-11-12T20:58:05.723ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
259612025-11-12T20:58:05.723ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
259622025-11-12T20:58:05.728ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ff9bf50-3627-41f2-9296-3ab41b6f1d16", "content-length": "682", "date": "Wed, 12 Nov 2025 20:58:05 GMT"} })
259632025-11-12T20:58:05.728ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259642025-11-12T20:58:05.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259652025-11-12T20:58:05.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259662025-11-12T20:58:05.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259672025-11-12T20:58:05.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259682025-11-12T20:58:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259692025-11-12T20:58:05.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
259702025-11-12T20:58:05.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
259712025-11-12T20:58:05.745ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 468.554946ms iteration = 26 status = {}
259722025-11-12T20:58:07.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 502 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 22073deb-fe4d-4112-89a8-433319f2793a response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
259732025-11-12T20:58:10.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 152 reason = Timeout
259742025-11-12T20:58:10.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
259752025-11-12T20:58:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
259762025-11-12T20:58:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
259772025-11-12T20:58:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
259782025-11-12T20:58:10.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
259792025-11-12T20:58:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
259802025-11-12T20:58:10.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
259812025-11-12T20:58:10.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.458653ms iteration = 152 status = {"config_updated":false}
259822025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
259832025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
259842025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
259852025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
259862025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 153 reason = Timeout
259872025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
259882025-11-12T20:58:15.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
259892025-11-12T20:58:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
259902025-11-12T20:58:15.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
259912025-11-12T20:58:15.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
259922025-11-12T20:58:15.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
259932025-11-12T20:58:15.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
259942025-11-12T20:58:15.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
259952025-11-12T20:58:15.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.46299ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
259962025-11-12T20:58:15.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
259972025-11-12T20:58:15.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
259982025-11-12T20:58:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
259992025-11-12T20:58:15.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260002025-11-12T20:58:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260012025-11-12T20:58:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260022025-11-12T20:58:15.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
260032025-11-12T20:58:15.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.435216ms iteration = 153 status = {"config_updated":false}
260042025-11-12T20:58:15.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
260052025-11-12T20:58:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260062025-11-12T20:58:15.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260072025-11-12T20:58:15.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
260082025-11-12T20:58:15.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
260092025-11-12T20:58:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260102025-11-12T20:58:15.300ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
260112025-11-12T20:58:15.300ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.119735ms iteration = 77 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
260122025-11-12T20:58:17.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 455 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 908bf97a-80cc-44a4-8c82-2803d71fd8d2 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
260132025-11-12T20:58:20.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 67 reason = Timeout
260142025-11-12T20:58:20.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
260152025-11-12T20:58:20.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
260162025-11-12T20:58:20.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 154 reason = Timeout
260172025-11-12T20:58:20.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260182025-11-12T20:58:20.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260192025-11-12T20:58:20.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260202025-11-12T20:58:20.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260212025-11-12T20:58:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
260222025-11-12T20:58:20.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
260232025-11-12T20:58:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
260242025-11-12T20:58:20.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260252025-11-12T20:58:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
260262025-11-12T20:58:20.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260272025-11-12T20:58:20.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 52 reason = Timeout
260282025-11-12T20:58:20.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
260292025-11-12T20:58:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
260302025-11-12T20:58:20.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
260312025-11-12T20:58:20.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.836023ms iteration = 154 status = {"config_updated":false}
260322025-11-12T20:58:20.284ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df7e6024-e0ad-4d74-ae9f-796a61ac3daf original_time_started = 2025-11-12 20:57:36.647597 UTC
260332025-11-12T20:58:20.284ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.544011ms iteration = 67 status = {"Loaded":{"collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","time_loaded":"2025-11-12T20:58:20.275812167Z","time_started":"2025-11-12T20:57:36.647597Z"}}
260342025-11-12T20:58:20.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260352025-11-12T20:58:20.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
260362025-11-12T20:58:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
260372025-11-12T20:58:20.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
260382025-11-12T20:58:20.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.862272ms iteration = 52 status = "NoSitrep"
260392025-11-12T20:58:20.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 52 reason = Timeout
260402025-11-12T20:58:20.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
260412025-11-12T20:58:20.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
260422025-11-12T20:58:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
260432025-11-12T20:58:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
260442025-11-12T20:58:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260452025-11-12T20:58:20.292ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
260462025-11-12T20:58:20.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
260472025-11-12T20:58:20.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
260482025-11-12T20:58:20.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
260492025-11-12T20:58:20.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
260502025-11-12T20:58:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260512025-11-12T20:58:20.297ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
260522025-11-12T20:58:20.298ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
260532025-11-12T20:58:20.298ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.29631ms iteration = 52 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
260542025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
260552025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
260562025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
260572025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 155 reason = Timeout
260582025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
260592025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260602025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
260612025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260622025-11-12T20:58:25.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260632025-11-12T20:58:25.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
260642025-11-12T20:58:25.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
260652025-11-12T20:58:25.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260662025-11-12T20:58:25.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260672025-11-12T20:58:25.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.140591ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
260682025-11-12T20:58:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
260692025-11-12T20:58:25.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
260702025-11-12T20:58:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260712025-11-12T20:58:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
260722025-11-12T20:58:25.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260732025-11-12T20:58:25.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260742025-11-12T20:58:25.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
260752025-11-12T20:58:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260762025-11-12T20:58:25.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
260772025-11-12T20:58:25.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.645308ms iteration = 155 status = {"config_updated":false}
260782025-11-12T20:58:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260792025-11-12T20:58:25.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
260802025-11-12T20:58:25.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
260812025-11-12T20:58:25.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
260822025-11-12T20:58:25.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
260832025-11-12T20:58:25.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.975872ms iteration = 78 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
260842025-11-12T20:58:27.937ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 586 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 543deed5-f762-4836-950a-55d517a5b1ce response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
260852025-11-12T20:58:30.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 156 reason = Timeout
260862025-11-12T20:58:30.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260872025-11-12T20:58:30.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260882025-11-12T20:58:30.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
260892025-11-12T20:58:30.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
260902025-11-12T20:58:30.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
260912025-11-12T20:58:30.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
260922025-11-12T20:58:30.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
260932025-11-12T20:58:30.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.140351ms iteration = 156 status = {"config_updated":false}
260942025-11-12T20:58:32.913ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:60626
260952025-11-12T20:58:32.913ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 0abb9e75-a380-4a6e-9f15-09435a17324e uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
260962025-11-12T20:58:32.913ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 0abb9e75-a380-4a6e-9f15-09435a17324e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
260972025-11-12T20:58:32.914ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 0abb9e75-a380-4a6e-9f15-09435a17324e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
260982025-11-12T20:58:32.916ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3436 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 0abb9e75-a380-4a6e-9f15-09435a17324e response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
260992025-11-12T20:58:32.917ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = b916de0f-d825-40a6-a5a1-c5b6fba4b989 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
261002025-11-12T20:58:32.917ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = b916de0f-d825-40a6-a5a1-c5b6fba4b989 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
261012025-11-12T20:58:32.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = b916de0f-d825-40a6-a5a1-c5b6fba4b989 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
261022025-11-12T20:58:32.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2579 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = b916de0f-d825-40a6-a5a1-c5b6fba4b989 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
261032025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
261042025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
261052025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
261062025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
261072025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
261082025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
261092025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
261102025-11-12T20:58:35.274ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-12 20:48:35.274267491 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
261112025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
261122025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
261132025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
261142025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
261152025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
261162025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
261172025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
261182025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
261192025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
261202025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
261212025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
261222025-11-12T20:58:35.274ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
261232025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
261242025-11-12T20:58:35.274ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = ExternalNetworking step id = Ensure
261252025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
261262025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
261272025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
261282025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
261292025-11-12T20:58:35.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
261302025-11-12T20:58:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0609a518-e1e4-4ce6-988e-4a3561afb1bd.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("825516cb-c031-4d35-9e79-628a18ad1e37.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("9549e270-af25-4b55-b470-cfd7b6fc49d2.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1710, tv_nsec: 772852636 } })
261312025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
261322025-11-12T20:58:35.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1ba3768a-f7d5-468c-84c2-9b8f16e59759.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8862ef13-79c3-4244-bc36-2b50e8268d5d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1710, tv_nsec: 772847216 } })
261332025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
261342025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
261352025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
261362025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 68 reason = Timeout
261372025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
261382025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
261392025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
261402025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
261412025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.495539ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
261422025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
261432025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
261442025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.463617ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353"]}
261452025-11-12T20:58:35.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
261462025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
261472025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
261482025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
261492025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
261502025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
261512025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 32 reason = Timeout
261522025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
261532025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
261542025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
261552025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
261562025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
261572025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 43 reason = Timeout
261582025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
261592025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
261602025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
261612025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
261622025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
261632025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
261642025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
261652025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
261662025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
261672025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
261682025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
261692025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
261702025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
261712025-11-12T20:58:35.276ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
261722025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
261732025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
261742025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
261752025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
261762025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
261772025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
261782025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
261792025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
261802025-11-12T20:58:35.276ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
261812025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 40 reason = Timeout
261822025-11-12T20:58:35.277ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
261832025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
261842025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
261852025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
261862025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
261872025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 157 reason = Timeout
261882025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
261892025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
261902025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
261912025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
261922025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 40 reason = Timeout
261932025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
261942025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
261952025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
261962025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
261972025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
261982025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
261992025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
262002025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
262012025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
262022025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
262032025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
262042025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
262052025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
262062025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
262072025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
262082025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
262092025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
262102025-11-12T20:58:35.277ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1710, tv_nsec: 775387112 } })
262112025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
262122025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
262132025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
262142025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
262152025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
262162025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
262172025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
262182025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
262192025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
262202025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
262212025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
262222025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
262232025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
262242025-11-12T20:58:35.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
262252025-11-12T20:58:35.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262262025-11-12T20:58:35.278ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-12 20:48:35.274267491 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
262272025-11-12T20:58:35.278ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262282025-11-12T20:58:35.278ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.369643ms iteration = 14 status = {"expiration":"2025-11-12T20:48:35.274267491Z","pruned":[]}
262292025-11-12T20:58:35.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262302025-11-12T20:58:35.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262312025-11-12T20:58:35.279ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1710, tv_nsec: 777363563 } })
262322025-11-12T20:58:35.279ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1710, tv_nsec: 777457296 } })
262332025-11-12T20:58:35.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
262342025-11-12T20:58:35.279ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
262352025-11-12T20:58:35.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
262362025-11-12T20:58:35.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
262372025-11-12T20:58:35.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.171558ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
262382025-11-12T20:58:35.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
262392025-11-12T20:58:35.281ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
262402025-11-12T20:58:35.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.751473ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
262412025-11-12T20:58:35.281ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.960738ms iteration = 14 status = {"by_rx":{},"error":null}
262422025-11-12T20:58:35.281ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
262432025-11-12T20:58:35.281ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
262442025-11-12T20:58:35.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
262452025-11-12T20:58:35.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262462025-11-12T20:58:35.282ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262472025-11-12T20:58:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
262482025-11-12T20:58:35.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
262492025-11-12T20:58:35.282ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262502025-11-12T20:58:35.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
262512025-11-12T20:58:35.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262522025-11-12T20:58:35.283ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262532025-11-12T20:58:35.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 53 reason = Timeout
262542025-11-12T20:58:35.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
262552025-11-12T20:58:35.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262562025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
262572025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262582025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
262592025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
262602025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 8.438449ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
262612025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262622025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
262632025-11-12T20:58:35.285ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262642025-11-12T20:58:35.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
262652025-11-12T20:58:35.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262662025-11-12T20:58:35.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
262672025-11-12T20:58:35.287ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
262682025-11-12T20:58:35.287ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.069227ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
262692025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 13.426727ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
262702025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
262712025-11-12T20:58:35.288ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
262722025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
262732025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
262742025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
262752025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
262762025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
262772025-11-12T20:58:35.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
262782025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
262792025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
262802025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
262812025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
262822025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
262832025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 53 reason = Timeout
262842025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
262852025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 14 reason = Timeout
262862025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
262872025-11-12T20:58:35.289ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
262882025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
262892025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
262902025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
262912025-11-12T20:58:35.289ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df7e6024-e0ad-4d74-ae9f-796a61ac3daf original_time_started = 2025-11-12 20:57:36.647597 UTC
262922025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 14.817833ms iteration = 68 status = {"Loaded":{"collection_id":"df7e6024-e0ad-4d74-ae9f-796a61ac3daf","time_loaded":"2025-11-12T20:58:35.275180405Z","time_started":"2025-11-12T20:57:36.647597Z"}}
262932025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
262942025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
262952025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
262962025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
262972025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
262982025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
262992025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
263002025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
263012025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
263022025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
263032025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
263042025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
263052025-11-12T20:58:35.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
263062025-11-12T20:58:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
263072025-11-12T20:58:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
263082025-11-12T20:58:35.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
263092025-11-12T20:58:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
263102025-11-12T20:58:35.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
263112025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263122025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
263132025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
263142025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
263152025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263162025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
263172025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
263182025-11-12T20:58:35.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
263192025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
263202025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
263212025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263222025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
263232025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
263242025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
263252025-11-12T20:58:35.293ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
263262025-11-12T20:58:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
263272025-11-12T20:58:35.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
263282025-11-12T20:58:35.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
263292025-11-12T20:58:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
263302025-11-12T20:58:35.295ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
263312025-11-12T20:58:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
263322025-11-12T20:58:35.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
263332025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
263342025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
263352025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
263362025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
263372025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
263382025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263392025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
263402025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
263412025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
263422025-11-12T20:58:35.297ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263432025-11-12T20:58:35.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
263442025-11-12T20:58:35.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
263452025-11-12T20:58:35.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
263462025-11-12T20:58:35.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
263472025-11-12T20:58:35.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263482025-11-12T20:58:35.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263492025-11-12T20:58:35.300ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263502025-11-12T20:58:35.300ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
263512025-11-12T20:58:35.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
263522025-11-12T20:58:35.303ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
263532025-11-12T20:58:35.303ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.604018ms iteration = 27 status = 0
263542025-11-12T20:58:35.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 19.832893ms iteration = 53 status = "NoSitrep"
263552025-11-12T20:58:35.304ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263562025-11-12T20:58:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
263572025-11-12T20:58:35.305ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
263582025-11-12T20:58:35.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
263592025-11-12T20:58:35.306ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
263602025-11-12T20:58:35.306ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1710, tv_nsec: 804727121 } })
263612025-11-12T20:58:35.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
263622025-11-12T20:58:35.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
263632025-11-12T20:58:35.307ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
263642025-11-12T20:58:35.307ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 30.120309ms iteration = 157 status = {"config_updated":false}
263652025-11-12T20:58:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
263662025-11-12T20:58:35.307ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
263672025-11-12T20:58:35.307ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 30.680824ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
263682025-11-12T20:58:35.307ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
263692025-11-12T20:58:35.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
263702025-11-12T20:58:35.308ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
263712025-11-12T20:58:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
263722025-11-12T20:58:35.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
263732025-11-12T20:58:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
263742025-11-12T20:58:35.309ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
263752025-11-12T20:58:35.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
263762025-11-12T20:58:35.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263772025-11-12T20:58:35.309ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
263782025-11-12T20:58:35.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
263792025-11-12T20:58:35.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
263802025-11-12T20:58:35.311ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
263812025-11-12T20:58:35.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
263822025-11-12T20:58:35.311ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: a32231ce-b4b9-448e-93c3-ba0de5d3f97e (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 1ba3768a-f7d5-468c-84c2-9b8f16e59759 zone_kind = external_dns
263832025-11-12T20:58:35.312ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263842025-11-12T20:58:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
263852025-11-12T20:58:35.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
263862025-11-12T20:58:35.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
263872025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
263882025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
263892025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
263902025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
263912025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
263922025-11-12T20:58:35.313ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
263932025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 36.959897ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
263942025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
263952025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
263962025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
263972025-11-12T20:58:35.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
263982025-11-12T20:58:35.315ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
263992025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
264002025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
264012025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
264022025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
264032025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
264042025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
264052025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264062025-11-12T20:58:35.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264072025-11-12T20:58:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
264082025-11-12T20:58:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
264092025-11-12T20:58:35.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
264102025-11-12T20:58:35.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
264112025-11-12T20:58:35.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
264122025-11-12T20:58:35.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
264132025-11-12T20:58:35.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264142025-11-12T20:58:35.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264152025-11-12T20:58:35.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
264162025-11-12T20:58:35.319ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
264172025-11-12T20:58:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
264182025-11-12T20:58:35.320ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264192025-11-12T20:58:35.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
264202025-11-12T20:58:35.320ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
264212025-11-12T20:58:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
264222025-11-12T20:58:35.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
264232025-11-12T20:58:35.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
264242025-11-12T20:58:35.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
264252025-11-12T20:58:35.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
264262025-11-12T20:58:35.322ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
264272025-11-12T20:58:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
264282025-11-12T20:58:35.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
264292025-11-12T20:58:35.324ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
264302025-11-12T20:58:35.324ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 33.016774ms iteration = 53 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
264312025-11-12T20:58:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
264322025-11-12T20:58:35.325ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
264332025-11-12T20:58:35.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
264342025-11-12T20:58:35.325ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
264352025-11-12T20:58:35.325ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 48.825963ms iteration = 40 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
264362025-11-12T20:58:35.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264372025-11-12T20:58:35.326ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 76429045-bc07-434d-ac80-fa0650b86ac4 (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 48173796-fa65-44a3-872c-519a0b1d8edc zone_kind = nexus
264382025-11-12T20:58:35.328ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
264392025-11-12T20:58:35.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
264402025-11-12T20:58:35.329ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
264412025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
264422025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
264432025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
264442025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
264452025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-12 20:46:04.924320 UTC dns_group = internal generation = 1
264462025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 55.368993ms iteration = 14 status = {"generation":1}
264472025-11-12T20:58:35.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
264482025-11-12T20:58:35.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
264492025-11-12T20:58:35.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
264502025-11-12T20:58:35.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
264512025-11-12T20:58:35.331ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264522025-11-12T20:58:35.331ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
264532025-11-12T20:58:35.331ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
264542025-11-12T20:58:35.331ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-12 20:46:05.686989 UTC dns_group = external generation = 2
264552025-11-12T20:58:35.331ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 57.503398ms iteration = 14 status = {"generation":2}
264562025-11-12T20:58:35.332ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 575be14e-614b-4912-aeca-53b295e68151 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 zone_kind = nexus
264572025-11-12T20:58:35.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
264582025-11-12T20:58:35.334ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
264592025-11-12T20:58:35.334ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
264602025-11-12T20:58:35.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
264612025-11-12T20:58:35.335ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
264622025-11-12T20:58:35.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
264632025-11-12T20:58:35.337ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 2440ad69-d041-4d17-bce7-e0eb0fa4e1e1 (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8862ef13-79c3-4244-bc36-2b50e8268d5d zone_kind = external_dns
264642025-11-12T20:58:35.337ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264652025-11-12T20:58:35.339ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
264662025-11-12T20:58:35.340ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: b6065695-2a13-4495-9ae0-4758c6dfc836 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 46bed455-6bb1-4379-9b59-23903d27e89e, kind: Service { id: 8b1ca59e-43cb-42c6-927b-4eb36698cfbb }, name: Name("ntp-8b1ca59e-43cb-42c6-927b-4eb36698cfbb"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb zone_kind = boundary_ntp
264672025-11-12T20:58:35.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
264682025-11-12T20:58:35.341ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
264692025-11-12T20:58:35.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
264702025-11-12T20:58:35.345ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
264712025-11-12T20:58:35.345ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264722025-11-12T20:58:35.345ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: c09ab55a-0d6a-41e6-8a0b-06e8daab8a86 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = d838cdc4-2fb9-4847-b11b-f37762feb37b zone_kind = nexus
264732025-11-12T20:58:35.346ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
264742025-11-12T20:58:35.346ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = SledList step id = Fetch
264752025-11-12T20:58:35.346ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 69.439257ms iteration = 79 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
264762025-11-12T20:58:35.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
264772025-11-12T20:58:35.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
264782025-11-12T20:58:35.348ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
264792025-11-12T20:58:35.348ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
264802025-11-12T20:58:35.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264812025-11-12T20:58:35.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
264822025-11-12T20:58:35.352ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
264832025-11-12T20:58:35.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
264842025-11-12T20:58:35.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
264852025-11-12T20:58:35.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264862025-11-12T20:58:35.359ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
264872025-11-12T20:58:35.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
264882025-11-12T20:58:35.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
264892025-11-12T20:58:35.368ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 91.307053ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
264902025-11-12T20:58:35.369ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = DeployNexusRecords step id = Ensure
264912025-11-12T20:58:35.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
264922025-11-12T20:58:35.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
264932025-11-12T20:58:35.376ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
264942025-11-12T20:58:35.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
264952025-11-12T20:58:35.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
264962025-11-12T20:58:35.376ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
264972025-11-12T20:58:35.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
264982025-11-12T20:58:35.379ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
264992025-11-12T20:58:35.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
265002025-11-12T20:58:35.379ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 102.76158ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
265012025-11-12T20:58:35.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
265022025-11-12T20:58:35.379ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
265032025-11-12T20:58:35.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
265042025-11-12T20:58:35.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
265052025-11-12T20:58:35.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
265062025-11-12T20:58:35.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265072025-11-12T20:58:35.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 107.844651ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
265082025-11-12T20:58:35.386ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
265092025-11-12T20:58:35.386ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
265102025-11-12T20:58:35.386ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 110.273773ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
265112025-11-12T20:58:35.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
265122025-11-12T20:58:35.390ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
265132025-11-12T20:58:35.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
265142025-11-12T20:58:35.392ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.884909ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
265152025-11-12T20:58:35.393ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
265162025-11-12T20:58:35.393ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
265172025-11-12T20:58:35.393ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 116.502775ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
265182025-11-12T20:58:35.402ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
265192025-11-12T20:58:35.402ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
265202025-11-12T20:58:35.402ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 125.246571ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
265212025-11-12T20:58:35.402ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 125.9975ms iteration = 43 status = {"physical_disks_added":0}
265222025-11-12T20:58:35.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265232025-11-12T20:58:35.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265242025-11-12T20:58:35.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265252025-11-12T20:58:35.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265262025-11-12T20:58:35.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265272025-11-12T20:58:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
265282025-11-12T20:58:35.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265292025-11-12T20:58:35.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265302025-11-12T20:58:35.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265312025-11-12T20:58:35.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265322025-11-12T20:58:35.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265332025-11-12T20:58:35.420ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-12T20:45:33.195604Z, time_modified: 2025-11-12T20:45:33.195604Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, db_silo: Silo { identity: SiloIdentity { id: 22a04497-95f6-4ebc-818b-f7ecb7a5e4b7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-12T20:46:05.618649Z, time_modified: 2025-11-12T20:46:05.618649Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd" }] }) }
265342025-11-12T20:58:35.420ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 145.66062ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]}},"default_endpoint":{"silo_id":"22a04497-95f6-4ebc-818b-f7ecb7a5e4b7","tls_certs":["fcd68a645e0a7e7d93d935b28f82ed56489d3472c02ff01103c61ecf65bd53dd"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
265352025-11-12T20:58:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
265362025-11-12T20:58:35.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265372025-11-12T20:58:35.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265382025-11-12T20:58:35.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265392025-11-12T20:58:35.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
265402025-11-12T20:58:35.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265412025-11-12T20:58:35.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265422025-11-12T20:58:35.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265432025-11-12T20:58:35.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265442025-11-12T20:58:35.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265452025-11-12T20:58:35.437ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
265462025-11-12T20:58:35.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265472025-11-12T20:58:35.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265482025-11-12T20:58:35.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265492025-11-12T20:58:35.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265502025-11-12T20:58:35.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265512025-11-12T20:58:35.452ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = SledAgent step id = Ensure
265522025-11-12T20:58:35.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
265532025-11-12T20:58:35.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265542025-11-12T20:58:35.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265552025-11-12T20:58:35.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265562025-11-12T20:58:35.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265572025-11-12T20:58:35.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265582025-11-12T20:58:35.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
265592025-11-12T20:58:35.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265602025-11-12T20:58:35.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265612025-11-12T20:58:35.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
265622025-11-12T20:58:35.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
265632025-11-12T20:58:35.490ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
265642025-11-12T20:58:35.490ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 213.806503ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
265652025-11-12T20:58:35.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265662025-11-12T20:58:35.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265672025-11-12T20:58:35.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265682025-11-12T20:58:35.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265692025-11-12T20:58:35.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265702025-11-12T20:58:35.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
265712025-11-12T20:58:35.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265722025-11-12T20:58:35.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265732025-11-12T20:58:35.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265742025-11-12T20:58:35.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
265752025-11-12T20:58:35.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265762025-11-12T20:58:35.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265772025-11-12T20:58:35.508ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
265782025-11-12T20:58:35.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265792025-11-12T20:58:35.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265802025-11-12T20:58:35.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265812025-11-12T20:58:35.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265822025-11-12T20:58:35.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265832025-11-12T20:58:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
265842025-11-12T20:58:35.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265852025-11-12T20:58:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265862025-11-12T20:58:35.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265872025-11-12T20:58:35.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
265882025-11-12T20:58:35.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265892025-11-12T20:58:35.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265902025-11-12T20:58:35.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265912025-11-12T20:58:35.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265922025-11-12T20:58:35.527ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
265932025-11-12T20:58:35.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265942025-11-12T20:58:35.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265952025-11-12T20:58:35.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
265962025-11-12T20:58:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
265972025-11-12T20:58:35.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
265982025-11-12T20:58:35.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
265992025-11-12T20:58:35.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266002025-11-12T20:58:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266012025-11-12T20:58:35.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
266022025-11-12T20:58:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
266032025-11-12T20:58:35.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266042025-11-12T20:58:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266052025-11-12T20:58:35.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266062025-11-12T20:58:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266072025-11-12T20:58:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
266082025-11-12T20:58:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
266092025-11-12T20:58:35.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266102025-11-12T20:58:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266112025-11-12T20:58:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266122025-11-12T20:58:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266132025-11-12T20:58:35.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
266142025-11-12T20:58:35.556ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
266152025-11-12T20:58:35.557ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 283.389773ms iteration = 27 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
266162025-11-12T20:58:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
266172025-11-12T20:58:35.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266182025-11-12T20:58:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266192025-11-12T20:58:35.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266202025-11-12T20:58:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266212025-11-12T20:58:35.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
266222025-11-12T20:58:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
266232025-11-12T20:58:35.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266242025-11-12T20:58:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266252025-11-12T20:58:35.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266262025-11-12T20:58:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266272025-11-12T20:58:35.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
266282025-11-12T20:58:35.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
266292025-11-12T20:58:35.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266302025-11-12T20:58:35.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266312025-11-12T20:58:35.582ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
266322025-11-12T20:58:35.582ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
266332025-11-12T20:58:35.582ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7193a76-e5d4-4608-9214-dbf58bcb521c", "content-length": "26", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
266342025-11-12T20:58:35.582ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
266352025-11-12T20:58:35.585ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
266362025-11-12T20:58:35.603ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
266372025-11-12T20:58:35.603ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
266382025-11-12T20:58:35.604ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfff635a-9094-4624-aacb-35033aef724d", "content-length": "372", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
266392025-11-12T20:58:35.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
266402025-11-12T20:58:35.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
266412025-11-12T20:58:35.619ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
266422025-11-12T20:58:35.619ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
266432025-11-12T20:58:35.619ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a3fcd76b-5e92-4dd5-a3f7-5ec84d7088a6", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
266442025-11-12T20:58:35.619ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
266452025-11-12T20:58:35.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
266462025-11-12T20:58:35.623ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
266472025-11-12T20:58:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
266482025-11-12T20:58:35.628ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
266492025-11-12T20:58:35.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
266502025-11-12T20:58:35.633ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
266512025-11-12T20:58:35.633ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
266522025-11-12T20:58:35.633ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f25b990b-bfda-4d3b-ab13-34871abebf1c", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
266532025-11-12T20:58:35.635ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 358.988881ms iteration = 40 status = {}
266542025-11-12T20:58:35.637ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
266552025-11-12T20:58:35.637ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
266562025-11-12T20:58:35.637ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 35a08948-d694-4aa8-be45-9389254a3e12 parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
266572025-11-12T20:58:35.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266582025-11-12T20:58:35.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266592025-11-12T20:58:35.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
266602025-11-12T20:58:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
266612025-11-12T20:58:35.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
266622025-11-12T20:58:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
266632025-11-12T20:58:35.645ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
266642025-11-12T20:58:35.645ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
266652025-11-12T20:58:35.645ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 369.107852ms iteration = 32 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
266662025-11-12T20:58:35.693ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
266672025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc8e38b1-4725-4db2-98fc-64a80619a516", "content-length": "78", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
266682025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
266692025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
266702025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
266712025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
266722025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
266732025-11-12T20:58:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
266742025-11-12T20:58:35.696ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
266752025-11-12T20:58:35.696ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
266762025-11-12T20:58:35.696ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
266772025-11-12T20:58:35.696ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
266782025-11-12T20:58:35.697ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e7b8b517-82a1-48fb-899f-1989d32175e6", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
266792025-11-12T20:58:35.697ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
266802025-11-12T20:58:35.698ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6bd6a226-6d1b-4e27-ab1f-e15b1ddb54a2", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
266812025-11-12T20:58:35.698ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
266822025-11-12T20:58:35.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
266832025-11-12T20:58:35.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
266842025-11-12T20:58:35.699ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
266852025-11-12T20:58:35.699ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
266862025-11-12T20:58:35.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
266872025-11-12T20:58:35.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
266882025-11-12T20:58:35.700ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
266892025-11-12T20:58:35.701ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
266902025-11-12T20:58:35.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
266912025-11-12T20:58:35.701ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
266922025-11-12T20:58:35.701ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 427.523584ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
266932025-11-12T20:58:35.701ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
266942025-11-12T20:58:35.701ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
266952025-11-12T20:58:35.702ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb5619bd-f4d8-4b48-824a-c06d6a8ee64a", "content-length": "26", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
266962025-11-12T20:58:35.702ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
266972025-11-12T20:58:35.703ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
266982025-11-12T20:58:35.703ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
266992025-11-12T20:58:35.703ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5cba855-006e-4ba1-9c94-38ca9a702436", "content-length": "81", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
267002025-11-12T20:58:35.703ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
267012025-11-12T20:58:35.704ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
267022025-11-12T20:58:35.722ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
267032025-11-12T20:58:35.723ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bba67ec-80cf-4fe7-9ab1-43356380316f", "content-length": "691", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
267042025-11-12T20:58:35.723ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
267052025-11-12T20:58:35.723ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
267062025-11-12T20:58:35.723ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
267072025-11-12T20:58:35.725ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 448.667591ms iteration = 27 status = {}
267082025-11-12T20:58:35.737ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
267092025-11-12T20:58:35.737ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
267102025-11-12T20:58:35.737ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "cda4b2ee-d33d-4fec-8c39-233016273f1d", "content-length": "81", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
267112025-11-12T20:58:35.746ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
267122025-11-12T20:58:35.746ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
267132025-11-12T20:58:35.746ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "07198da0-7391-4702-8b68-e803ee46680b", "content-length": "81", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
267142025-11-12T20:58:35.748ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 471.02967ms iteration = 27 status = {"disabled":false,"errors":[],"sps":[]}
267152025-11-12T20:58:35.767ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
267162025-11-12T20:58:35.768ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
267172025-11-12T20:58:35.769ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "faa602d3-b160-4436-8bea-ba393efbabf4", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} }) sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
267182025-11-12T20:58:35.769ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "f27198a1-430b-478c-a56e-29b2a4bb6e37", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
267192025-11-12T20:58:35.769ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
267202025-11-12T20:58:35.771ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
267212025-11-12T20:58:35.771ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 481.547342ms iteration = 14 status = {"errors":[],"probes_by_sled":{"cdf575e5-7c27-4aa3-b1ec-77468da13285":0}}
267222025-11-12T20:58:35.771ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 41 reason = Dependency
267232025-11-12T20:58:35.771ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
267242025-11-12T20:58:35.771ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
267252025-11-12T20:58:35.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
267262025-11-12T20:58:35.772ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = FirewallRules step id = Ensure
267272025-11-12T20:58:35.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267282025-11-12T20:58:35.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267292025-11-12T20:58:35.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
267302025-11-12T20:58:35.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267312025-11-12T20:58:35.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267322025-11-12T20:58:35.775ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
267332025-11-12T20:58:35.775ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
267342025-11-12T20:58:35.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
267352025-11-12T20:58:35.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267362025-11-12T20:58:35.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267372025-11-12T20:58:35.777ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
267382025-11-12T20:58:35.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267392025-11-12T20:58:35.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267402025-11-12T20:58:35.780ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
267412025-11-12T20:58:35.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
267422025-11-12T20:58:35.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
267432025-11-12T20:58:35.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267442025-11-12T20:58:35.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267452025-11-12T20:58:35.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267462025-11-12T20:58:35.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267472025-11-12T20:58:35.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
267482025-11-12T20:58:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
267492025-11-12T20:58:35.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267502025-11-12T20:58:35.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267512025-11-12T20:58:35.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267522025-11-12T20:58:35.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267532025-11-12T20:58:35.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267542025-11-12T20:58:35.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267552025-11-12T20:58:35.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
267562025-11-12T20:58:35.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
267572025-11-12T20:58:35.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267582025-11-12T20:58:35.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267592025-11-12T20:58:35.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267602025-11-12T20:58:35.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267612025-11-12T20:58:35.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267622025-11-12T20:58:35.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267632025-11-12T20:58:35.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267642025-11-12T20:58:35.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267652025-11-12T20:58:35.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267662025-11-12T20:58:35.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267672025-11-12T20:58:35.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267682025-11-12T20:58:35.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267692025-11-12T20:58:35.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267702025-11-12T20:58:35.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267712025-11-12T20:58:35.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
267722025-11-12T20:58:35.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
267732025-11-12T20:58:35.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267742025-11-12T20:58:35.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267752025-11-12T20:58:35.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267762025-11-12T20:58:35.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267772025-11-12T20:58:35.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267782025-11-12T20:58:35.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267792025-11-12T20:58:35.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267802025-11-12T20:58:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267812025-11-12T20:58:35.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267822025-11-12T20:58:35.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267832025-11-12T20:58:35.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267842025-11-12T20:58:35.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267852025-11-12T20:58:35.826ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
267862025-11-12T20:58:35.826ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1711, tv_nsec: 324619575 } })
267872025-11-12T20:58:35.826ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "b6129659-673f-4d45-9120-a43d33de2fc5", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267882025-11-12T20:58:35.826ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267892025-11-12T20:58:35.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
267902025-11-12T20:58:35.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
267912025-11-12T20:58:35.828ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ccd4c7fd-73d0-4f7e-a72e-819cf37ab477", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267922025-11-12T20:58:35.828ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267932025-11-12T20:58:35.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
267942025-11-12T20:58:35.830ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "3fac3b8d-df78-4bd5-ac2e-8f0a05216a46", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267952025-11-12T20:58:35.830ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267962025-11-12T20:58:35.832ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
267972025-11-12T20:58:35.832ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 558.704439ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
267982025-11-12T20:58:35.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
267992025-11-12T20:58:35.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268002025-11-12T20:58:35.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268012025-11-12T20:58:35.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268022025-11-12T20:58:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268032025-11-12T20:58:35.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268042025-11-12T20:58:35.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268052025-11-12T20:58:35.838ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 564.55884ms iteration = 27 status = {}
268062025-11-12T20:58:35.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
268072025-11-12T20:58:35.844ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
268082025-11-12T20:58:35.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268092025-11-12T20:58:35.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268102025-11-12T20:58:35.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268112025-11-12T20:58:35.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268122025-11-12T20:58:35.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268132025-11-12T20:58:35.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268142025-11-12T20:58:35.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268152025-11-12T20:58:35.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268162025-11-12T20:58:35.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268172025-11-12T20:58:35.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268182025-11-12T20:58:35.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268192025-11-12T20:58:35.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268202025-11-12T20:58:35.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268212025-11-12T20:58:35.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268222025-11-12T20:58:35.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268232025-11-12T20:58:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268242025-11-12T20:58:35.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
268252025-11-12T20:58:35.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
268262025-11-12T20:58:35.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268272025-11-12T20:58:35.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268282025-11-12T20:58:35.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268292025-11-12T20:58:35.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268302025-11-12T20:58:35.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268312025-11-12T20:58:35.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268322025-11-12T20:58:35.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
268332025-11-12T20:58:35.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
268342025-11-12T20:58:35.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268352025-11-12T20:58:35.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268362025-11-12T20:58:35.874ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
268372025-11-12T20:58:35.875ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
268382025-11-12T20:58:35.879ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a355d57-ee0c-48a1-99af-1900c867b641", "content-length": "372", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
268392025-11-12T20:58:35.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
268402025-11-12T20:58:35.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
268412025-11-12T20:58:35.880ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
268422025-11-12T20:58:35.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268432025-11-12T20:58:35.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268442025-11-12T20:58:35.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
268452025-11-12T20:58:35.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
268462025-11-12T20:58:35.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268472025-11-12T20:58:35.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268482025-11-12T20:58:35.886ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
268492025-11-12T20:58:35.886ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 55140af2-abd2-43ae-b049-b6684a1a68fe, kind: Service { id: 1ba3768a-f7d5-468c-84c2-9b8f16e59759 }, name: Name("external-dns-1ba3768a-f7d5-468c-84c2-9b8f16e59759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 7900bc93-41cf-49a4-bdc5-a81db77999a8, kind: Service { id: 8862ef13-79c3-4244-bc36-2b50e8268d5d }, name: Name("external-dns-8862ef13-79c3-4244-bc36-2b50e8268d5d"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 29220e5e-baed-49df-b4c1-9feb778d74d3, kind: Service { id: 48173796-fa65-44a3-872c-519a0b1d8edc }, name: Name("nexus-48173796-fa65-44a3-872c-519a0b1d8edc"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 197])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 4d643693-bacd-4b2a-9ff0-ed3748dcab13, kind: Service { id: d838cdc4-2fb9-4847-b11b-f37762feb37b }, name: Name("nexus-d838cdc4-2fb9-4847-b11b-f37762feb37b"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 29])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 8070cf86-0f6f-4502-a4c1-948b683412aa, kind: Service { id: 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85 }, name: Name("nexus-6e9a1534-f78a-4e09-bd1d-d1ea863f3c85"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 215])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
268502025-11-12T20:58:35.886ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
268512025-11-12T20:58:35.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
268522025-11-12T20:58:35.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
268532025-11-12T20:58:35.919ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
268542025-11-12T20:58:35.919ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
268552025-11-12T20:58:35.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
268562025-11-12T20:58:35.928ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
268572025-11-12T20:58:35.928ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
268582025-11-12T20:58:35.929ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d5794522-236b-464e-85a7-a14d7e3bcf18", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
268592025-11-12T20:58:35.929ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
268602025-11-12T20:58:35.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
268612025-11-12T20:58:35.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
268622025-11-12T20:58:35.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
268632025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
268642025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
268652025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
268662025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
268672025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
268682025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
268692025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
268702025-11-12T20:58:35.936ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
268712025-11-12T20:58:35.937ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
268722025-11-12T20:58:35.937ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
268732025-11-12T20:58:35.937ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
268742025-11-12T20:58:35.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
268752025-11-12T20:58:35.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
268762025-11-12T20:58:35.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
268772025-11-12T20:58:35.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
268782025-11-12T20:58:35.943ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
268792025-11-12T20:58:35.943ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
268802025-11-12T20:58:35.944ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
268812025-11-12T20:58:35.944ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
268822025-11-12T20:58:35.951ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a839ee54-be03-4230-840c-9e17eafa16b8", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
268832025-11-12T20:58:35.953ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 181.695021ms iteration = 41 status = {}
268842025-11-12T20:58:35.953ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), time_created: 2025-11-12T20:46:00.902032Z, time_modified: 2025-11-12T20:46:00.902032Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
268852025-11-12T20:58:35.953ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
268862025-11-12T20:58:35.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268872025-11-12T20:58:35.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268882025-11-12T20:58:35.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268892025-11-12T20:58:35.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268902025-11-12T20:58:35.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
268912025-11-12T20:58:35.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
268922025-11-12T20:58:35.993ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
268932025-11-12T20:58:35.996ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "b09a1e9d-c1f5-4cf6-bb85-2f6c29159143", "date": "Wed, 12 Nov 2025 20:58:35 GMT"} })
268942025-11-12T20:58:35.998ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
268952025-11-12T20:58:35.999ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = Dns step id = Ensure
268962025-11-12T20:58:35.999ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
268972025-11-12T20:58:35.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
268982025-11-12T20:58:35.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
268992025-11-12T20:58:35.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269002025-11-12T20:58:36.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269012025-11-12T20:58:36.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269022025-11-12T20:58:36.006ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269032025-11-12T20:58:36.042ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
269042025-11-12T20:58:36.042ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
269052025-11-12T20:58:36.042ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
269062025-11-12T20:58:36.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269072025-11-12T20:58:36.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269082025-11-12T20:58:36.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269092025-11-12T20:58:36.048ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269102025-11-12T20:58:36.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269112025-11-12T20:58:36.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269122025-11-12T20:58:36.050ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
269132025-11-12T20:58:36.050ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
269142025-11-12T20:58:36.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269152025-11-12T20:58:36.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269162025-11-12T20:58:36.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269172025-11-12T20:58:36.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269182025-11-12T20:58:36.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269192025-11-12T20:58:36.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269202025-11-12T20:58:36.059ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 159b361c-3732-4b29-a173-0c72005cbdae dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
269212025-11-12T20:58:36.059ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
269222025-11-12T20:58:36.059ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
269232025-11-12T20:58:36.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269242025-11-12T20:58:36.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269252025-11-12T20:58:36.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269262025-11-12T20:58:36.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269272025-11-12T20:58:36.061ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "335e6024-627c-4f3f-b235-c855e565c932", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
269282025-11-12T20:58:36.061ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
269292025-11-12T20:58:36.062ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
269302025-11-12T20:58:36.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269312025-11-12T20:58:36.062ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b67a680-274f-4cbc-8a47-4c29467c51ff", "content-length": "682", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
269322025-11-12T20:58:36.062ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
269332025-11-12T20:58:36.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
269342025-11-12T20:58:36.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269352025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
269362025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
269372025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
269382025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
269392025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269402025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
269412025-11-12T20:58:36.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269422025-11-12T20:58:36.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269432025-11-12T20:58:36.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
269442025-11-12T20:58:36.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
269452025-11-12T20:58:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269462025-11-12T20:58:36.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269472025-11-12T20:58:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269482025-11-12T20:58:36.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
269492025-11-12T20:58:36.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
269502025-11-12T20:58:36.074ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
269512025-11-12T20:58:36.074ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 0 version = 2
269522025-11-12T20:58:36.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269532025-11-12T20:58:36.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269542025-11-12T20:58:36.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269552025-11-12T20:58:36.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269562025-11-12T20:58:36.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269572025-11-12T20:58:36.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269582025-11-12T20:58:36.080ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 803.955595ms iteration = 27 status = {}
269592025-11-12T20:58:36.082ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 990e9fa8-76db-43e2-a76c-a2fb785a1d88 dns_zone_name = oxide.test found_so_far = 4 version = 2
269602025-11-12T20:58:36.082ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
269612025-11-12T20:58:36.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269622025-11-12T20:58:36.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269632025-11-12T20:58:36.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269642025-11-12T20:58:36.089ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
269652025-11-12T20:58:36.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269662025-11-12T20:58:36.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269672025-11-12T20:58:36.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269682025-11-12T20:58:36.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269692025-11-12T20:58:36.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269702025-11-12T20:58:36.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269712025-11-12T20:58:36.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269722025-11-12T20:58:36.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
269732025-11-12T20:58:36.099ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
269742025-11-12T20:58:36.099ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
269752025-11-12T20:58:36.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = OmicronZones step id = Cleanup
269762025-11-12T20:58:36.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = OmicronZones step id = Cleanup
269772025-11-12T20:58:36.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = PhysicalDisks step id = Cleanup
269782025-11-12T20:58:36.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = Clickhouse step id = Ensure
269792025-11-12T20:58:36.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = Clickhouse step id = Ensure
269802025-11-12T20:58:36.133ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
269812025-11-12T20:58:36.138ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "82f0979b-8208-4292-a0b6-1e4814b644a0", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
269822025-11-12T20:58:36.140ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = SupportBundles step id = Cleanup
269832025-11-12T20:58:36.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269842025-11-12T20:58:36.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269852025-11-12T20:58:36.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269862025-11-12T20:58:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
269872025-11-12T20:58:36.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269882025-11-12T20:58:36.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269892025-11-12T20:58:36.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269902025-11-12T20:58:36.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269912025-11-12T20:58:36.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269922025-11-12T20:58:36.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
269932025-11-12T20:58:36.155ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = OmicronZones step id = Cleanup
269942025-11-12T20:58:36.155ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
269952025-11-12T20:58:36.155ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
269962025-11-12T20:58:36.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
269972025-11-12T20:58:36.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
269982025-11-12T20:58:36.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
269992025-11-12T20:58:36.158ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
270002025-11-12T20:58:36.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
270012025-11-12T20:58:36.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
270022025-11-12T20:58:36.194ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
270032025-11-12T20:58:36.194ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(ed21ad55-76d9-4726-8fdf-e6b9c9e170df (blueprint), false)
270042025-11-12T20:58:36.194ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = Cockroach step id = Ensure
270052025-11-12T20:58:36.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
270062025-11-12T20:58:36.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
270072025-11-12T20:58:36.215ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
270082025-11-12T20:58:36.215ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1a6509e6-107d-42bb-9570-ff47ac047789 step component = MgsUpdates step id = Ensure
270092025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 41 reason = Dependency
270102025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Dependency
270112025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
270122025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
270132025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
270142025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
270152025-11-12T20:58:36.215ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 941.110724ms iteration = 15 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3930,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":71862089,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":71862029,"secs":0}},"event_index":1,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":71876761,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":23564259,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":23564210,"secs":0}},"event_index":2,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":95451991,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":83010948,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":83010898,"secs":0}},"event_index":3,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":178482159,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":319654162,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":319654112,"secs":0}},"event_index":4,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":498160392,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":226416818,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":226416778,"secs":0}},"event_index":5,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":724590471,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":100374218,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":100374158,"secs":0}},"event_index":6,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":824973619,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4521,"secs":0}},"event_index":7,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":824992670,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2121,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2081,"secs":0}},"event_index":8,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":824997340,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3370,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3330,"secs":0}},"event_index":9,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":825002690,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1620,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1590,"secs":0}},"event_index":10,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":825006400,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40567040,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":40567000,"secs":0}},"event_index":11,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":865574960,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15377698,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15377668,"secs":0}},"event_index":12,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":880964838,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38853116,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":38853066,"secs":0}},"event_index":13,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":919828924,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":20775648,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":20775598,"secs":0}},"event_index":14,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":940614772,"secs":0}},{"data":{"attempt_elapsed":{"nanos":7820,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":7780,"secs":0}},"event_index":15,"execution_id":"596e9243-1784-483c-941b-ee56bf4712a9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":940632922,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df"}
270162025-11-12T20:58:36.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
270172025-11-12T20:58:36.217ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
270182025-11-12T20:58:36.220ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
270192025-11-12T20:58:36.220ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
270202025-11-12T20:58:36.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
270212025-11-12T20:58:36.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
270222025-11-12T20:58:36.221ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
270232025-11-12T20:58:36.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
270242025-11-12T20:58:36.223ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
270252025-11-12T20:58:36.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
270262025-11-12T20:58:36.223ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(98e4220e-c01e-4ccf-adc3-06f59cdfaa04, 0), (0db01969-2d5f-44e1-a7c7-53fbadaf8cf1, 0), (82bc89d4-4300-49cc-99b1-9c95524e2299, 0), (df7e6024-e0ad-4d74-ae9f-796a61ac3daf, 0)] collection_id = 98e4220e-c01e-4ccf-adc3-06f59cdfaa04
270272025-11-12T20:58:36.223ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
270282025-11-12T20:58:36.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
270292025-11-12T20:58:36.225ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
270302025-11-12T20:58:36.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
270312025-11-12T20:58:36.227ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
270322025-11-12T20:58:36.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
270332025-11-12T20:58:36.229ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
270342025-11-12T20:58:36.229ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 14.811814ms iteration = 41 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
270352025-11-12T20:58:36.403ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 98e4220e-c01e-4ccf-adc3-06f59cdfaa04 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
270362025-11-12T20:58:36.403ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
270372025-11-12T20:58:36.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
270382025-11-12T20:58:36.408ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(0db01969-2d5f-44e1-a7c7-53fbadaf8cf1, 0), (82bc89d4-4300-49cc-99b1-9c95524e2299, 0), (df7e6024-e0ad-4d74-ae9f-796a61ac3daf, 0)]
270392025-11-12T20:58:36.409ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1711, tv_nsec: 907272483 } })
270402025-11-12T20:58:36.443ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("05f2607d-3a97-4234-aa15-3a6fd8df0992.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("2ec2e9f6-28cb-49f1-a8fd-50881409a1c2.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("41e62f52-5b3a-44bc-be0f-be1f355d8c97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5c4b3ca5-f698-45fe-b064-96cb04df0316.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fac217cf-dbe9-43c9-8ca3-716a02fd1669.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1711, tv_nsec: 941549600 } })
270412025-11-12T20:58:36.444ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
270422025-11-12T20:58:36.444ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
270432025-11-12T20:58:36.444ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
270442025-11-12T20:58:36.444ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "304132f4-4e35-4bfc-a5e3-08e6a87aa194", "content-length": "476", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270452025-11-12T20:58:36.444ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
270462025-11-12T20:58:36.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a999cc-63d7-495a-a8ff-f68867b504e4", "content-length": "734", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270472025-11-12T20:58:36.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
270482025-11-12T20:58:36.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1195e5d2-cdcc-4d19-bbfe-6c4ad6d213ef", "content-length": "111", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270492025-11-12T20:58:36.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
270502025-11-12T20:58:36.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c0547cf-9a4b-4393-a677-d342a1f04427", "content-length": "111", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270512025-11-12T20:58:36.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
270522025-11-12T20:58:36.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c772169-4464-4b39-a27e-7ca1e3666343", "content-length": "163", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270532025-11-12T20:58:36.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
270542025-11-12T20:58:36.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aceffa10-6dd6-458a-b6bc-3b04888ba4dc", "content-length": "163", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270552025-11-12T20:58:36.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
270562025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "588ca90c-9740-4599-abd6-3fa6b3923f38", "content-length": "166", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270572025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
270582025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "88160bfd-66ef-4d5b-9645-c4adc891b743", "content-length": "166", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270592025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
270602025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d9cb21d-b337-4e84-a55c-09d53e95f7c1", "content-length": "702", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270612025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
270622025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bbbeebb-b998-49a0-8180-53e6f25f8f1e", "content-length": "727", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270632025-11-12T20:58:36.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
270642025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b806ef2-e1aa-4684-a476-c992fe03cdef", "content-length": "729", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270652025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
270662025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd41e31d-505c-4e93-9e8a-b56e144f7ac4", "content-length": "728", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270672025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
270682025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6191672-e7f6-4016-a3a1-1f527a9cf85d", "content-length": "722", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270692025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
270702025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a64437d-2767-4d42-ada3-aabbc2632cab", "content-length": "10", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270712025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
270722025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bd5dd01-33fe-4a5c-8334-da7791b56830", "content-length": "144", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270732025-11-12T20:58:36.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
270742025-11-12T20:58:36.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e32172e3-18ae-413f-8b29-134711bdc1f2", "content-length": "147", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270752025-11-12T20:58:36.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
270762025-11-12T20:58:36.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ab8cf2-76fa-4383-b3ad-a2d1286ecd71", "content-length": "109", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270772025-11-12T20:58:36.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
270782025-11-12T20:58:36.449ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c856113e-1801-4ee8-8535-f333f0d3c168", "content-length": "109", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270792025-11-12T20:58:36.449ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
270802025-11-12T20:58:36.450ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d1db19b-a1de-40a7-8a70-361308e7cd32", "content-length": "163", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270812025-11-12T20:58:36.450ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
270822025-11-12T20:58:36.450ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "260f524a-2f93-44d1-a651-63f55cb0b61b", "content-length": "163", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270832025-11-12T20:58:36.450ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
270842025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "12b175da-472d-409a-b9bb-cdfa14a06e55", "content-length": "166", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270852025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
270862025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a34bb3-3a1f-4bc0-97ea-d719e050f5e3", "content-length": "166", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270872025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
270882025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "68764b47-9e03-4b70-ad10-d996d0996291", "content-length": "702", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270892025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
270902025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e78ad720-b008-4e4e-8c0f-a8584a8d0a99", "content-length": "727", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270912025-11-12T20:58:36.451ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
270922025-11-12T20:58:36.452ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da28e5f8-0a6e-42d2-8165-4e5b3c58028f", "content-length": "729", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270932025-11-12T20:58:36.452ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
270942025-11-12T20:58:36.452ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f30be4c-54c4-43e9-a1bf-9088bae26c27", "content-length": "728", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
270952025-11-12T20:58:36.452ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
270962025-11-12T20:58:36.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
270972025-11-12T20:58:36.454ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
270982025-11-12T20:58:36.456ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
270992025-11-12T20:58:36.456ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
271002025-11-12T20:58:36.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
271012025-11-12T20:58:36.458ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271022025-11-12T20:58:36.460ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
271032025-11-12T20:58:36.460ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
271042025-11-12T20:58:36.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
271052025-11-12T20:58:36.496ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
271062025-11-12T20:58:36.496ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
271072025-11-12T20:58:36.579ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d85c984-57f1-4afc-a686-972b1fb20a71", "content-length": "95384", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271082025-11-12T20:58:36.581ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
271092025-11-12T20:58:36.581ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
271102025-11-12T20:58:36.581ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
271112025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
271122025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
271132025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
271142025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
271152025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
271162025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a00b8741-9e1f-4efa-8666-22511c1fb1db", "content-length": "64", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271172025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
271182025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec7de799-aa8a-4988-b421-21d9982ad392", "content-length": "64", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271192025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b710a18-1619-4c23-91a1-10a9a64d0912", "content-length": "64", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271202025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
271212025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
271222025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc12036-813c-4784-b051-cfdaf9829c95", "content-length": "64", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271232025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "69c0972a-e6f0-4fef-b0ac-01ac006d04b4", "content-length": "64", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271242025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
271252025-11-12T20:58:36.582ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
271262025-11-12T20:58:36.620ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cccc56dc-d5f3-4883-81fe-abb4a68bc0e8", "content-length": "419530", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271272025-11-12T20:58:36.621ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "561a1aa4-f389-4447-b61b-26e4e0277aed", "content-length": "414592", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271282025-11-12T20:58:36.623ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1251292-861a-4ba8-aec4-e81844ed5712", "content-length": "438011", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271292025-11-12T20:58:36.624ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "60527a91-3d3e-4987-b446-549b159ca30d", "content-length": "409948", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271302025-11-12T20:58:36.625ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a612811-9750-482c-8047-bcdf99118044", "content-length": "416192", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271312025-11-12T20:58:36.627ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
271322025-11-12T20:58:36.627ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
271332025-11-12T20:58:36.627ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
271342025-11-12T20:58:36.627ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
271352025-11-12T20:58:36.627ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
271362025-11-12T20:58:36.667ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 8b1ca59e-43cb-42c6-927b-4eb36698cfbb (omicron_zone)
271372025-11-12T20:58:36.667ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
271382025-11-12T20:58:36.671ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "abc79e34-612a-41c4-abe1-5e4749ef989f", "content-length": "124", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271392025-11-12T20:58:36.673ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
271402025-11-12T20:58:36.769ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
271412025-11-12T20:58:36.769ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
271422025-11-12T20:58:36.769ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd2f1cfe-df1b-4cae-8f1b-13dffd1671ef", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271432025-11-12T20:58:36.770ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0609a518-e1e4-4ce6-988e-4a3561afb1bd (omicron_zone)
271442025-11-12T20:58:36.771ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
271452025-11-12T20:58:36.771ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
271462025-11-12T20:58:36.772ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6733ff54-b357-4640-bfa3-9774d25c5a1d", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271472025-11-12T20:58:36.772ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 825516cb-c031-4d35-9e79-628a18ad1e37 (omicron_zone)
271482025-11-12T20:58:36.773ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
271492025-11-12T20:58:36.773ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
271502025-11-12T20:58:36.774ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "906aa551-2e9e-4ab3-987c-dd2787b1296b", "content-length": "9552", "date": "Wed, 12 Nov 2025 20:58:36 GMT"} })
271512025-11-12T20:58:36.774ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 9549e270-af25-4b55-b470-cfd7b6fc49d2 (omicron_zone)
271522025-11-12T20:58:36.776ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
271532025-11-12T20:58:36.776ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
271542025-11-12T20:58:36.778ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
271552025-11-12T20:58:36.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
271562025-11-12T20:58:36.781ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271572025-11-12T20:58:36.783ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
271582025-11-12T20:58:36.783ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
271592025-11-12T20:58:36.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
271602025-11-12T20:58:36.972ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = a5446e5c-ff31-4004-9e03-19d9245abaa4 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
271612025-11-12T20:58:36.972ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = a5446e5c-ff31-4004-9e03-19d9245abaa4 time_started = 2025-11-12 20:58:36.444226 UTC
271622025-11-12T20:58:36.972ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 69 reason = Dependency
271632025-11-12T20:58:36.972ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
271642025-11-12T20:58:36.972ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 757.527974ms iteration = 16 status = {"collection_id":"a5446e5c-ff31-4004-9e03-19d9245abaa4","time_done":"2025-11-12 20:58:36.776434 UTC","time_started":"2025-11-12 20:58:36.444226 UTC"}
271652025-11-12T20:58:36.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
271662025-11-12T20:58:36.974ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271672025-11-12T20:58:36.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
271682025-11-12T20:58:36.978ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
271692025-11-12T20:58:36.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
271702025-11-12T20:58:37.110ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
271712025-11-12T20:58:37.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
271722025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 179.503135ms iteration = 69 status = {"Loaded":{"collection_id":"5ab14e54-3449-40be-a07f-463d924a139a","time_loaded":"2025-11-12T20:58:36.972559792Z","time_started":"2025-11-12T20:58:36.496864Z"}}
271732025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 44 reason = Dependency
271742025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 20 reason = Dependency
271752025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 33 reason = Dependency
271762025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
271772025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
271782025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
271792025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
271802025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
271812025-11-12T20:58:37.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
271822025-11-12T20:58:37.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271832025-11-12T20:58:37.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271842025-11-12T20:58:37.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271852025-11-12T20:58:37.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
271862025-11-12T20:58:37.156ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
271872025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
271882025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
271892025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
271902025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
271912025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
271922025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
271932025-11-12T20:58:37.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
271942025-11-12T20:58:37.158ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
271952025-11-12T20:58:37.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
271962025-11-12T20:58:37.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271972025-11-12T20:58:37.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271982025-11-12T20:58:37.160ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
271992025-11-12T20:58:37.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
272002025-11-12T20:58:37.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272012025-11-12T20:58:37.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
272022025-11-12T20:58:37.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272032025-11-12T20:58:37.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272042025-11-12T20:58:37.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272052025-11-12T20:58:37.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
272062025-11-12T20:58:37.163ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
272072025-11-12T20:58:37.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
272082025-11-12T20:58:37.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272092025-11-12T20:58:37.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272102025-11-12T20:58:37.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272112025-11-12T20:58:37.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272122025-11-12T20:58:37.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272132025-11-12T20:58:37.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272142025-11-12T20:58:37.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
272152025-11-12T20:58:37.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272162025-11-12T20:58:37.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
272172025-11-12T20:58:37.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272182025-11-12T20:58:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272192025-11-12T20:58:37.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272202025-11-12T20:58:37.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272212025-11-12T20:58:37.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272222025-11-12T20:58:37.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
272232025-11-12T20:58:37.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272242025-11-12T20:58:37.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272252025-11-12T20:58:37.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
272262025-11-12T20:58:37.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272272025-11-12T20:58:37.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272282025-11-12T20:58:37.175ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
272292025-11-12T20:58:37.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272302025-11-12T20:58:37.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272312025-11-12T20:58:37.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272322025-11-12T20:58:37.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272332025-11-12T20:58:37.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272342025-11-12T20:58:37.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272352025-11-12T20:58:37.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272362025-11-12T20:58:37.180ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
272372025-11-12T20:58:37.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272382025-11-12T20:58:37.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272392025-11-12T20:58:37.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272402025-11-12T20:58:37.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272412025-11-12T20:58:37.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
272422025-11-12T20:58:37.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
272432025-11-12T20:58:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
272442025-11-12T20:58:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
272452025-11-12T20:58:37.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272462025-11-12T20:58:37.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272472025-11-12T20:58:37.186ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
272482025-11-12T20:58:37.186ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 34.388559ms iteration = 20 status = {"blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","inventory_collection_id":"5ab14e54-3449-40be-a07f-463d924a139a","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
272492025-11-12T20:58:37.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272502025-11-12T20:58:37.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272512025-11-12T20:58:37.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272522025-11-12T20:58:37.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
272532025-11-12T20:58:37.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272542025-11-12T20:58:37.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272552025-11-12T20:58:37.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272562025-11-12T20:58:37.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
272572025-11-12T20:58:37.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272582025-11-12T20:58:37.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272592025-11-12T20:58:37.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272602025-11-12T20:58:37.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272612025-11-12T20:58:37.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272622025-11-12T20:58:37.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8724fb5f-2806-4206-a142-e910ec94ebd0, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
272632025-11-12T20:58:37.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272642025-11-12T20:58:37.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272652025-11-12T20:58:37.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272662025-11-12T20:58:37.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272672025-11-12T20:58:37.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272682025-11-12T20:58:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
272692025-11-12T20:58:37.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272702025-11-12T20:58:37.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272712025-11-12T20:58:37.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272722025-11-12T20:58:37.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272732025-11-12T20:58:37.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272742025-11-12T20:58:37.211ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 59.701845ms iteration = 44 status = {"physical_disks_added":0}
272752025-11-12T20:58:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
272762025-11-12T20:58:37.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272772025-11-12T20:58:37.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272782025-11-12T20:58:37.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272792025-11-12T20:58:37.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272802025-11-12T20:58:37.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272812025-11-12T20:58:37.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272822025-11-12T20:58:37.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272832025-11-12T20:58:37.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
272842025-11-12T20:58:37.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272852025-11-12T20:58:37.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272862025-11-12T20:58:37.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272872025-11-12T20:58:37.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
272882025-11-12T20:58:37.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272892025-11-12T20:58:37.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272902025-11-12T20:58:37.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272912025-11-12T20:58:37.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272922025-11-12T20:58:37.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272932025-11-12T20:58:37.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272942025-11-12T20:58:37.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272952025-11-12T20:58:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
272962025-11-12T20:58:37.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
272972025-11-12T20:58:37.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
272982025-11-12T20:58:37.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272992025-11-12T20:58:37.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 8f4e2cb4-2ca2-477c-867e-1dd6ff9fb625, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
273002025-11-12T20:58:37.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273012025-11-12T20:58:37.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273022025-11-12T20:58:37.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273032025-11-12T20:58:37.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273042025-11-12T20:58:37.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273052025-11-12T20:58:37.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273062025-11-12T20:58:37.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273072025-11-12T20:58:37.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
273082025-11-12T20:58:37.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273092025-11-12T20:58:37.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273102025-11-12T20:58:37.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273112025-11-12T20:58:37.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273122025-11-12T20:58:37.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273132025-11-12T20:58:37.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
273142025-11-12T20:58:37.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273152025-11-12T20:58:37.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273162025-11-12T20:58:37.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273172025-11-12T20:58:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273182025-11-12T20:58:37.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273192025-11-12T20:58:37.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
273202025-11-12T20:58:37.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273212025-11-12T20:58:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273222025-11-12T20:58:37.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273232025-11-12T20:58:37.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273242025-11-12T20:58:37.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273252025-11-12T20:58:37.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
273262025-11-12T20:58:37.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273272025-11-12T20:58:37.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273282025-11-12T20:58:37.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273292025-11-12T20:58:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273302025-11-12T20:58:37.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273312025-11-12T20:58:37.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
273322025-11-12T20:58:37.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273332025-11-12T20:58:37.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273342025-11-12T20:58:37.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273352025-11-12T20:58:37.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273362025-11-12T20:58:37.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273372025-11-12T20:58:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
273382025-11-12T20:58:37.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273392025-11-12T20:58:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273402025-11-12T20:58:37.296ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
273412025-11-12T20:58:37.296ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
273422025-11-12T20:58:37.296ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 268cad8b-670a-4639-87d8-27434e4c900b parent_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df sled_id = cdf575e5-7c27-4aa3-b1ec-77468da13285
273432025-11-12T20:58:37.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273442025-11-12T20:58:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273452025-11-12T20:58:37.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273462025-11-12T20:58:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
273472025-11-12T20:58:37.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
273482025-11-12T20:58:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
273492025-11-12T20:58:37.304ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
273502025-11-12T20:58:37.304ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df
273512025-11-12T20:58:37.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 152.425494ms iteration = 33 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled cdf575e5-7c27-4aa3-b1ec-77468da13285: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
273522025-11-12T20:58:37.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 377 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 300ab888-fba6-4022-a777-6f3b2a572c10 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
273532025-11-12T20:58:40.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 158 reason = Timeout
273542025-11-12T20:58:40.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
273552025-11-12T20:58:40.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
273562025-11-12T20:58:40.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273572025-11-12T20:58:40.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
273582025-11-12T20:58:40.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
273592025-11-12T20:58:40.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
273602025-11-12T20:58:40.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
273612025-11-12T20:58:40.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.478013ms iteration = 158 status = {"config_updated":false}
273622025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
273632025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
273642025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
273652025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
273662025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 159 reason = Timeout
273672025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
273682025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
273692025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
273702025-11-12T20:58:45.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
273712025-11-12T20:58:45.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273722025-11-12T20:58:45.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
273732025-11-12T20:58:45.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273742025-11-12T20:58:45.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
273752025-11-12T20:58:45.280ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.961213ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
273762025-11-12T20:58:45.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
273772025-11-12T20:58:45.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
273782025-11-12T20:58:45.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
273792025-11-12T20:58:45.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
273802025-11-12T20:58:45.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
273812025-11-12T20:58:45.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
273822025-11-12T20:58:45.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
273832025-11-12T20:58:45.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.094944ms iteration = 159 status = {"config_updated":false}
273842025-11-12T20:58:45.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
273852025-11-12T20:58:45.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
273862025-11-12T20:58:45.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
273872025-11-12T20:58:45.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
273882025-11-12T20:58:45.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
273892025-11-12T20:58:45.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
273902025-11-12T20:58:45.306ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
273912025-11-12T20:58:45.306ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.969356ms iteration = 80 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
273922025-11-12T20:58:47.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 309 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = e37ff6e3-c25e-4516-95a4-d7154a810d7e response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
273932025-11-12T20:58:50.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 70 reason = Timeout
273942025-11-12T20:58:50.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
273952025-11-12T20:58:50.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
273962025-11-12T20:58:50.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 160 reason = Timeout
273972025-11-12T20:58:50.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
273982025-11-12T20:58:50.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
273992025-11-12T20:58:50.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274002025-11-12T20:58:50.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274012025-11-12T20:58:50.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
274022025-11-12T20:58:50.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
274032025-11-12T20:58:50.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
274042025-11-12T20:58:50.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
274052025-11-12T20:58:50.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
274062025-11-12T20:58:50.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
274072025-11-12T20:58:50.282ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5ab14e54-3449-40be-a07f-463d924a139a original_time_started = 2025-11-12 20:58:36.496864 UTC
274082025-11-12T20:58:50.282ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 6.94384ms iteration = 70 status = {"Loaded":{"collection_id":"5ab14e54-3449-40be-a07f-463d924a139a","time_loaded":"2025-11-12T20:58:50.275362997Z","time_started":"2025-11-12T20:58:36.496864Z"}}
274092025-11-12T20:58:50.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 54 reason = Timeout
274102025-11-12T20:58:50.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
274112025-11-12T20:58:50.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
274122025-11-12T20:58:50.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.118204ms iteration = 160 status = {"config_updated":false}
274132025-11-12T20:58:50.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
274142025-11-12T20:58:50.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274152025-11-12T20:58:50.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
274162025-11-12T20:58:50.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
274172025-11-12T20:58:50.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
274182025-11-12T20:58:50.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.527923ms iteration = 54 status = "NoSitrep"
274192025-11-12T20:58:50.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 54 reason = Timeout
274202025-11-12T20:58:50.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
274212025-11-12T20:58:50.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
274222025-11-12T20:58:50.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
274232025-11-12T20:58:50.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
274242025-11-12T20:58:50.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274252025-11-12T20:58:50.293ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
274262025-11-12T20:58:50.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
274272025-11-12T20:58:50.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
274282025-11-12T20:58:50.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
274292025-11-12T20:58:50.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
274302025-11-12T20:58:50.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274312025-11-12T20:58:50.298ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
274322025-11-12T20:58:50.299ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
274332025-11-12T20:58:50.299ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.984788ms iteration = 54 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
274342025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
274352025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
274362025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
274372025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
274382025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 161 reason = Timeout
274392025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
274402025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
274412025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
274422025-11-12T20:58:55.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
274432025-11-12T20:58:55.279ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
274442025-11-12T20:58:55.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274452025-11-12T20:58:55.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274462025-11-12T20:58:55.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
274472025-11-12T20:58:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
274482025-11-12T20:58:55.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
274492025-11-12T20:58:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
274502025-11-12T20:58:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
274512025-11-12T20:58:55.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
274522025-11-12T20:58:55.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
274532025-11-12T20:58:55.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
274542025-11-12T20:58:55.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
274552025-11-12T20:58:55.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.379328ms iteration = 161 status = {"config_updated":false}
274562025-11-12T20:58:55.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
274572025-11-12T20:58:55.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274582025-11-12T20:58:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
274592025-11-12T20:58:55.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
274602025-11-12T20:58:55.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
274612025-11-12T20:58:55.303ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
274622025-11-12T20:58:55.303ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.164997ms iteration = 81 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
274632025-11-12T20:58:55.313ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.518685ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
274642025-11-12T20:58:57.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 428 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 28462f19-1cf1-4d65-a8ab-2aa21a3268a2 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
274652025-11-12T20:59:00.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 162 reason = Timeout
274662025-11-12T20:59:00.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
274672025-11-12T20:59:00.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
274682025-11-12T20:59:00.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
274692025-11-12T20:59:00.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
274702025-11-12T20:59:00.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
274712025-11-12T20:59:00.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
274722025-11-12T20:59:00.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
274732025-11-12T20:59:00.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.134195ms iteration = 162 status = {"config_updated":false}
274742025-11-12T20:59:02.913ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 6dfaedec-d838-43ca-97b5-29fda31bcbb5 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending version = 1.0.0
274752025-11-12T20:59:02.913ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 6dfaedec-d838-43ca-97b5-29fda31bcbb5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
274762025-11-12T20:59:02.913ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 6dfaedec-d838-43ca-97b5-29fda31bcbb5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
274772025-11-12T20:59:02.916ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2813 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 6dfaedec-d838-43ca-97b5-29fda31bcbb5 response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?sort_by=id_ascending
274782025-11-12T20:59:02.916ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 79bc4cd6-d642-4a94-84f8-91caa655be6b uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D version = 1.0.0
274792025-11-12T20:59:02.916ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 79bc4cd6-d642-4a94-84f8-91caa655be6b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
274802025-11-12T20:59:02.916ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 79bc4cd6-d642-4a94-84f8-91caa655be6b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
274812025-11-12T20:59:02.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2845 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:60626 req_id = 79bc4cd6-d642-4a94-84f8-91caa655be6b response_code = 200 uri = /metrics/collectors/ecb0f932-e2e8-41d3-85a9-494e5cae5bce/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZmMxYzY0Yy0zODBkLTQ3MmMtOTYxZi05ZTlmNjZmZjJjZTIifX0%3D
274822025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
274832025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 28 reason = Timeout
274842025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
274852025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
274862025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
274872025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
274882025-11-12T20:59:05.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
274892025-11-12T20:59:05.275ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 71 reason = Timeout
274902025-11-12T20:59:05.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
274912025-11-12T20:59:05.275ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
274922025-11-12T20:59:05.275ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
274932025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
274942025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
274952025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
274962025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
274972025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
274982025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
274992025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 28 reason = Timeout
275002025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
275012025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
275022025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
275032025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
275042025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
275052025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 45 reason = Timeout
275062025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
275072025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
275082025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
275092025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
275102025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
275112025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
275122025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
275132025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
275142025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
275152025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
275162025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
275172025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
275182025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 42 reason = Timeout
275192025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
275202025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
275212025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
275222025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 163 reason = Timeout
275232025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
275242025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
275252025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 42 reason = Timeout
275262025-11-12T20:59:05.276ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
275272025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
275282025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
275292025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
275302025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
275312025-11-12T20:59:05.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 774741573 } })
275322025-11-12T20:59:05.276ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 774736903 } })
275332025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
275342025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
275352025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
275362025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
275372025-11-12T20:59:05.276ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 774830806 } })
275382025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
275392025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
275402025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
275412025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
275422025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
275432025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
275442025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
275452025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
275462025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
275472025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
275482025-11-12T20:59:05.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
275492025-11-12T20:59:05.277ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
275502025-11-12T20:59:05.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275512025-11-12T20:59:05.278ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275522025-11-12T20:59:05.279ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275532025-11-12T20:59:05.279ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.036756ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
275542025-11-12T20:59:05.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275552025-11-12T20:59:05.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275562025-11-12T20:59:05.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275572025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275582025-11-12T20:59:05.280ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
275592025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
275602025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
275612025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
275622025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
275632025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
275642025-11-12T20:59:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
275652025-11-12T20:59:05.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 55 reason = Timeout
275662025-11-12T20:59:05.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
275672025-11-12T20:59:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
275682025-11-12T20:59:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
275692025-11-12T20:59:05.284ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
275702025-11-12T20:59:05.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
275712025-11-12T20:59:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
275722025-11-12T20:59:05.285ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
275732025-11-12T20:59:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
275742025-11-12T20:59:05.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
275752025-11-12T20:59:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
275762025-11-12T20:59:05.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
275772025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
275782025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
275792025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
275802025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
275812025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
275822025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
275832025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
275842025-11-12T20:59:05.287ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
275852025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
275862025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
275872025-11-12T20:59:05.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
275882025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
275892025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
275902025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 12.325019ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
275912025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
275922025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
275932025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
275942025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5ab14e54-3449-40be-a07f-463d924a139a original_time_started = 2025-11-12 20:58:36.496864 UTC
275952025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.307535ms iteration = 71 status = {"Loaded":{"collection_id":"5ab14e54-3449-40be-a07f-463d924a139a","time_loaded":"2025-11-12T20:59:05.275488078Z","time_started":"2025-11-12T20:58:36.496864Z"}}
275962025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
275972025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
275982025-11-12T20:59:05.288ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
275992025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276002025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276012025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
276022025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
276032025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 55 reason = Timeout
276042025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
276052025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
276062025-11-12T20:59:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
276072025-11-12T20:59:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
276082025-11-12T20:59:05.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
276092025-11-12T20:59:05.290ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276102025-11-12T20:59:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
276112025-11-12T20:59:05.291ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 789333541 } })
276122025-11-12T20:59:05.291ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
276132025-11-12T20:59:05.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
276142025-11-12T20:59:05.291ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
276152025-11-12T20:59:05.291ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 15.566713ms iteration = 163 status = {"config_updated":false}
276162025-11-12T20:59:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
276172025-11-12T20:59:05.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276182025-11-12T20:59:05.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
276192025-11-12T20:59:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
276202025-11-12T20:59:05.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276212025-11-12T20:59:05.293ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 17.170444ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
276222025-11-12T20:59:05.293ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
276232025-11-12T20:59:05.293ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 17.744829ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
276242025-11-12T20:59:05.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
276252025-11-12T20:59:05.294ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
276262025-11-12T20:59:05.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
276272025-11-12T20:59:05.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276282025-11-12T20:59:05.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276292025-11-12T20:59:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: cdf575e5-7c27-4aa3-b1ec-77468da13285 (sled), lookup_type: ById(cdf575e5-7c27-4aa3-b1ec-77468da13285) } result = Ok(())
276302025-11-12T20:59:05.296ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 13.027238ms iteration = 55 status = "NoSitrep"
276312025-11-12T20:59:05.296ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
276322025-11-12T20:59:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
276332025-11-12T20:59:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
276342025-11-12T20:59:05.297ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
276352025-11-12T20:59:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
276362025-11-12T20:59:05.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
276372025-11-12T20:59:05.297ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
276382025-11-12T20:59:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
276392025-11-12T20:59:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
276402025-11-12T20:59:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
276412025-11-12T20:59:05.299ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
276422025-11-12T20:59:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
276432025-11-12T20:59:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
276442025-11-12T20:59:05.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
276452025-11-12T20:59:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
276462025-11-12T20:59:05.301ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.249243ms iteration = 28 status = 0
276472025-11-12T20:59:05.301ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
276482025-11-12T20:59:05.301ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 25.558822ms iteration = 42 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
276492025-11-12T20:59:05.304ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
276502025-11-12T20:59:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
276512025-11-12T20:59:05.305ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
276522025-11-12T20:59:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
276532025-11-12T20:59:05.308ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
276542025-11-12T20:59:05.310ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
276552025-11-12T20:59:05.311ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
276562025-11-12T20:59:05.311ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 22.203885ms iteration = 55 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
276572025-11-12T20:59:05.311ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
276582025-11-12T20:59:05.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
276592025-11-12T20:59:05.337ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
276602025-11-12T20:59:05.337ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 61.145663ms iteration = 82 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
276612025-11-12T20:59:05.348ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
276622025-11-12T20:59:05.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
276632025-11-12T20:59:05.349ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.179393ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
276642025-11-12T20:59:05.350ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.655082ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
276652025-11-12T20:59:05.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
276662025-11-12T20:59:05.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
276672025-11-12T20:59:05.358ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
276682025-11-12T20:59:05.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
276692025-11-12T20:59:05.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 83.285095ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
276702025-11-12T20:59:05.361ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
276712025-11-12T20:59:05.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
276722025-11-12T20:59:05.363ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 87.143365ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
276732025-11-12T20:59:05.368ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
276742025-11-12T20:59:05.368ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
276752025-11-12T20:59:05.368ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 92.118624ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
276762025-11-12T20:59:05.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 94.252059ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
276772025-11-12T20:59:05.395ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
276782025-11-12T20:59:05.395ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
276792025-11-12T20:59:05.396ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a7e835a-9714-4d31-a1c2-c0b67f14da7c", "content-length": "26", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
276802025-11-12T20:59:05.396ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
276812025-11-12T20:59:05.398ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
276822025-11-12T20:59:05.398ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
276832025-11-12T20:59:05.398ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "84e2fb68-56ea-41d4-9984-a66c3863b99a", "content-length": "78", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
276842025-11-12T20:59:05.398ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
276852025-11-12T20:59:05.398ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
276862025-11-12T20:59:05.398ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
276872025-11-12T20:59:05.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
276882025-11-12T20:59:05.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
276892025-11-12T20:59:05.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
276902025-11-12T20:59:05.402ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
276912025-11-12T20:59:05.402ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
276922025-11-12T20:59:05.405ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
276932025-11-12T20:59:05.405ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
276942025-11-12T20:59:05.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
276952025-11-12T20:59:05.406ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
276962025-11-12T20:59:05.406ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
276972025-11-12T20:59:05.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
276982025-11-12T20:59:05.409ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 26a1c56f-e508-433d-a013-f9b1f7f7b6b0 (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
276992025-11-12T20:59:05.409ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2
277002025-11-12T20:59:05.409ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=26a1c56f-e508-433d-a013-f9b1f7f7b6b0&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c13b0de-5729-4b63-ac31-5c474125e0ae", "content-length": "81", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277012025-11-12T20:59:05.411ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 zone_address = fd00:1122:3344:101::2
277022025-11-12T20:59:05.411ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
277032025-11-12T20:59:05.411ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
277042025-11-12T20:59:05.411ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f605101-d3ae-4b30-82a1-9588c213d2bd", "content-length": "26", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277052025-11-12T20:59:05.411ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
277062025-11-12T20:59:05.414ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 140.910107ms iteration = 28 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
277072025-11-12T20:59:05.414ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
277082025-11-12T20:59:05.449ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = b2948518-afb3-433a-bf05-8c84e6450ab2 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
277092025-11-12T20:59:05.449ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2
277102025-11-12T20:59:05.450ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=b2948518-afb3-433a-bf05-8c84e6450ab2&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "28f4f959-1e89-48a0-8ca8-747f6f281d28", "content-length": "81", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277112025-11-12T20:59:05.452ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
277122025-11-12T20:59:05.454ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
277132025-11-12T20:59:05.454ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
277142025-11-12T20:59:05.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
277152025-11-12T20:59:05.457ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 73b1da76-28ff-42e0-98dc-eaa405d92765 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
277162025-11-12T20:59:05.457ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2
277172025-11-12T20:59:05.457ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=73b1da76-28ff-42e0-98dc-eaa405d92765&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "77d4d380-7409-440e-8510-99583b09e05f", "content-length": "81", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277182025-11-12T20:59:05.459ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 183.54298ms iteration = 28 status = {"disabled":false,"errors":[],"sps":[]}
277192025-11-12T20:59:05.466ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 190.023087ms iteration = 45 status = {"physical_disks_added":0}
277202025-11-12T20:59:05.476ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
277212025-11-12T20:59:05.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
277222025-11-12T20:59:05.482ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
277232025-11-12T20:59:05.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
277242025-11-12T20:59:05.485ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
277252025-11-12T20:59:05.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
277262025-11-12T20:59:05.487ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
277272025-11-12T20:59:05.488ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-cdf575e5-7c27-4aa3-b1ec-77468da13285.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 986236487 } })
277282025-11-12T20:59:05.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
277292025-11-12T20:59:05.489ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
277302025-11-12T20:59:05.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
277312025-11-12T20:59:05.496ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 222.997211ms iteration = 28 status = {}
277322025-11-12T20:59:05.500ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
277332025-11-12T20:59:05.500ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
277342025-11-12T20:59:05.503ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d6fe619-feb5-4cc2-a188-58c81e87d747", "content-length": "372", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277352025-11-12T20:59:05.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
277362025-11-12T20:59:05.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
277372025-11-12T20:59:05.551ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
277382025-11-12T20:59:05.551ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
277392025-11-12T20:59:05.552ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "86b5f4fe-8669-4f7a-a600-83bddd71a366", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277402025-11-12T20:59:05.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
277412025-11-12T20:59:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
277422025-11-12T20:59:05.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
277432025-11-12T20:59:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
277442025-11-12T20:59:05.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
277452025-11-12T20:59:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
277462025-11-12T20:59:05.561ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277472025-11-12T20:59:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277482025-11-12T20:59:05.562ZINFOnexus (BackgroundTasks): VPC route manager sled cdf575e5-7c27-4aa3-b1ec-77468da13285 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
277492025-11-12T20:59:05.562ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
277502025-11-12T20:59:05.563ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c82369ed-0a7d-4d68-b237-594927f69af7", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277512025-11-12T20:59:05.563ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 5f8fe45c-905e-49c6-a03a-75b43a8e2525, rack_id: 21dec677-5bdd-45c9-88cb-9a589af920a7, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2a110015-db93-4485-8fc5-a7e6c947022f) } rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
277522025-11-12T20:59:05.563ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277532025-11-12T20:59:05.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277542025-11-12T20:59:05.565ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 289.79962ms iteration = 42 status = {}
277552025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
277562025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2a110015-db93-4485-8fc5-a7e6c947022f
277572025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
277582025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
277592025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
277602025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
277612025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
277622025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
277632025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
277642025-11-12T20:59:05.575ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
277652025-11-12T20:59:05.576ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7
277662025-11-12T20:59:05.576ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277672025-11-12T20:59:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277682025-11-12T20:59:05.579ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch_location = Switch0
277692025-11-12T20:59:05.579ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
277702025-11-12T20:59:05.598ZDEBGnexus (BackgroundTasks): client request SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
277712025-11-12T20:59:05.601ZDEBGnexus (BackgroundTasks): client response SledAgent = cdf575e5-7c27-4aa3-b1ec-77468da13285 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c4ef6db-37da-454e-8673-a2b58469e0a9", "content-length": "691", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277722025-11-12T20:59:05.601ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
277732025-11-12T20:59:05.601ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
277742025-11-12T20:59:05.601ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
277752025-11-12T20:59:05.603ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 327.408083ms iteration = 28 status = {}
277762025-11-12T20:59:05.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "701b6817-9622-457d-8ec7-c455e8e9538b", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277772025-11-12T20:59:05.693ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 switch = Switch0
277782025-11-12T20:59:05.693ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
277792025-11-12T20:59:05.697ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 21dec677-5bdd-45c9-88cb-9a589af920a7 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a25fb3db-5414-4b4f-b158-d2a46f0a0e11", "content-length": "682", "date": "Wed, 12 Nov 2025 20:59:05 GMT"} })
277802025-11-12T20:59:05.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277812025-11-12T20:59:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277822025-11-12T20:59:05.701ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277832025-11-12T20:59:05.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277842025-11-12T20:59:05.704ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277852025-11-12T20:59:05.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277862025-11-12T20:59:05.707ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
277872025-11-12T20:59:05.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
277882025-11-12T20:59:05.714ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 438.296762ms iteration = 28 status = {}
277892025-11-12T20:59:06.336ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 5
277902025-11-12T20:59:06.336ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
277912025-11-12T20:59:06.368ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
277922025-11-12T20:59:06.368ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:58028
277932025-11-12T20:59:06.369ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58028 req_id = 7d453365-b07b-4d46-9026-88dcb9ad15b8 uri = /metrics/producers version = 1.0.0
277942025-11-12T20:59:06.369ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58028 req_id = 7d453365-b07b-4d46-9026-88dcb9ad15b8 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
277952025-11-12T20:59:06.369ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58028 req_id = 7d453365-b07b-4d46-9026-88dcb9ad15b8 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
277962025-11-12T20:59:06.416ZINFOnexus: assigned collector to new producer collector_id = ecb0f932-e2e8-41d3-85a9-494e5cae5bce file = nexus/src/app/oximeter.rs:76 producer_id = 6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
277972025-11-12T20:59:06.416ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 47884 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:58028 req_id = 7d453365-b07b-4d46-9026-88dcb9ad15b8 response_code = 201 uri = /metrics/producers
277982025-11-12T20:59:06.417ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "7d453365-b07b-4d46-9026-88dcb9ad15b8", "content-length": "41", "date": "Wed, 12 Nov 2025 20:59:06 GMT"} })
277992025-11-12T20:59:06.419ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
278002025-11-12T20:59:07.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 308 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = c67c37fe-792a-4a86-958c-1f0194bfdd91 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
278012025-11-12T20:59:10.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 164 reason = Timeout
278022025-11-12T20:59:10.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278032025-11-12T20:59:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278042025-11-12T20:59:10.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278052025-11-12T20:59:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
278062025-11-12T20:59:10.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278072025-11-12T20:59:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278082025-11-12T20:59:10.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
278092025-11-12T20:59:10.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.542295ms iteration = 164 status = {"config_updated":false}
278102025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
278112025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
278122025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
278132025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
278142025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 165 reason = Timeout
278152025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278162025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
278172025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
278182025-11-12T20:59:15.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278192025-11-12T20:59:15.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278202025-11-12T20:59:15.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278212025-11-12T20:59:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
278222025-11-12T20:59:15.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
278232025-11-12T20:59:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
278242025-11-12T20:59:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
278252025-11-12T20:59:15.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278262025-11-12T20:59:15.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278272025-11-12T20:59:15.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
278282025-11-12T20:59:15.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
278292025-11-12T20:59:15.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
278302025-11-12T20:59:15.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.155432ms iteration = 165 status = {"config_updated":false}
278312025-11-12T20:59:15.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278322025-11-12T20:59:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
278332025-11-12T20:59:15.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
278342025-11-12T20:59:15.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
278352025-11-12T20:59:15.304ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
278362025-11-12T20:59:15.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.793159ms iteration = 83 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
278372025-11-12T20:59:15.310ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
278382025-11-12T20:59:15.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
278392025-11-12T20:59:15.312ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.891039ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
278402025-11-12T20:59:17.935ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 295 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 3a31353e-6939-4a5b-9733-6cc32c27cc8c response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
278412025-11-12T20:59:20.274ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 72 reason = Timeout
278422025-11-12T20:59:20.274ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
278432025-11-12T20:59:20.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
278442025-11-12T20:59:20.275ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 166 reason = Timeout
278452025-11-12T20:59:20.275ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278462025-11-12T20:59:20.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278472025-11-12T20:59:20.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278482025-11-12T20:59:20.280ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278492025-11-12T20:59:20.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
278502025-11-12T20:59:20.281ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
278512025-11-12T20:59:20.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
278522025-11-12T20:59:20.283ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5ab14e54-3449-40be-a07f-463d924a139a original_time_started = 2025-11-12 20:58:36.496864 UTC
278532025-11-12T20:59:20.283ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 56 reason = Timeout
278542025-11-12T20:59:20.283ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.424588ms iteration = 72 status = {"Loaded":{"collection_id":"5ab14e54-3449-40be-a07f-463d924a139a","time_loaded":"2025-11-12T20:59:20.274854043Z","time_started":"2025-11-12T20:58:36.496864Z"}}
278552025-11-12T20:59:20.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
278562025-11-12T20:59:20.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
278572025-11-12T20:59:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
278582025-11-12T20:59:20.284ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278592025-11-12T20:59:20.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278602025-11-12T20:59:20.284ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278612025-11-12T20:59:20.286ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
278622025-11-12T20:59:20.286ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.245715ms iteration = 166 status = {"config_updated":false}
278632025-11-12T20:59:20.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
278642025-11-12T20:59:20.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
278652025-11-12T20:59:20.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
278662025-11-12T20:59:20.288ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.217815ms iteration = 56 status = "NoSitrep"
278672025-11-12T20:59:20.289ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 56 reason = Timeout
278682025-11-12T20:59:20.289ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
278692025-11-12T20:59:20.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
278702025-11-12T20:59:20.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
278712025-11-12T20:59:20.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
278722025-11-12T20:59:20.291ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278732025-11-12T20:59:20.292ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
278742025-11-12T20:59:20.295ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
278752025-11-12T20:59:20.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
278762025-11-12T20:59:20.296ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
278772025-11-12T20:59:20.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
278782025-11-12T20:59:20.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278792025-11-12T20:59:20.299ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
278802025-11-12T20:59:20.299ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
278812025-11-12T20:59:20.299ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.556933ms iteration = 56 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
278822025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
278832025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
278842025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
278852025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
278862025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 167 reason = Timeout
278872025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278882025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
278892025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
278902025-11-12T20:59:25.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278912025-11-12T20:59:25.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278922025-11-12T20:59:25.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
278932025-11-12T20:59:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
278942025-11-12T20:59:25.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
278952025-11-12T20:59:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
278962025-11-12T20:59:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
278972025-11-12T20:59:25.282ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
278982025-11-12T20:59:25.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
278992025-11-12T20:59:25.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
279002025-11-12T20:59:25.284ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
279012025-11-12T20:59:25.284ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.371527ms iteration = 167 status = {"config_updated":false}
279022025-11-12T20:59:25.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
279032025-11-12T20:59:25.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
279042025-11-12T20:59:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: ed21ad55-76d9-4726-8fdf-e6b9c9e170df, lookup_type: ById(ed21ad55-76d9-4726-8fdf-e6b9c9e170df) } result = Ok(())
279052025-11-12T20:59:25.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
279062025-11-12T20:59:25.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
279072025-11-12T20:59:25.301ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df original_time_created = 2025-11-12 20:45:30.502476 UTC target_id = ed21ad55-76d9-4726-8fdf-e6b9c9e170df time_created = 2025-11-12 20:45:30.502476 UTC
279082025-11-12T20:59:25.301ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.249653ms iteration = 84 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"ed21ad55-76d9-4726-8fdf-e6b9c9e170df","time_created":"2025-11-12T20:45:30.502476Z"}
279092025-11-12T20:59:25.311ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
279102025-11-12T20:59:25.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
279112025-11-12T20:59:25.312ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.360341ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
279122025-11-12T20:59:27.936ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 335 local_addr = [fd00:1122:3344:101::c]:46805 method = GET remote_addr = [fd00:1122:3344:101::d]:38173 req_id = 1d9b2cb2-5a63-44ce-bbce-6ce48e808ea0 response_code = 200 uri = /6e9a1534-f78a-4e09-bd1d-d1ea863f3c85
279132025-11-12T20:59:30.276ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 168 reason = Timeout
279142025-11-12T20:59:30.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
279152025-11-12T20:59:30.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
279162025-11-12T20:59:30.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
279172025-11-12T20:59:30.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
279182025-11-12T20:59:30.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
279192025-11-12T20:59:30.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
279202025-11-12T20:59:30.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
279212025-11-12T20:59:30.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.31266ms iteration = 168 status = {"config_updated":false}